-- Logs begin at Thu 2022-06-16 16:55:59 UTC, end at Thu 2022-06-16 18:05:28 UTC. -- Jun 16 17:29:01.888246 dv-cinder-837542-4-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jun 16 17:29:01.901466 dv-cinder-837542-4-iscsi devstack@keystone.service[80346]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jun 16 17:29:01.902826 dv-cinder-837542-4-iscsi devstack@keystone.service[80346]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Jun 16 17:29:01 2022] *** Jun 16 17:29:01.902826 dv-cinder-837542-4-iscsi devstack@keystone.service[80346]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jun 16 17:29:01.902826 dv-cinder-837542-4-iscsi devstack@keystone.service[80346]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jun 16 17:29:01.902826 dv-cinder-837542-4-iscsi devstack@keystone.service[80346]: nodename: dv-cinder-837542-4-iscsi Jun 16 17:29:01.902826 dv-cinder-837542-4-iscsi devstack@keystone.service[80346]: machine: x86_64 Jun 16 17:29:01.902826 dv-cinder-837542-4-iscsi devstack@keystone.service[80346]: clock source: unix Jun 16 17:29:01.902826 dv-cinder-837542-4-iscsi devstack@keystone.service[80346]: pcre jit disabled Jun 16 17:29:01.902826 dv-cinder-837542-4-iscsi devstack@keystone.service[80346]: detected number of CPU cores: 4 Jun 16 17:29:01.902826 dv-cinder-837542-4-iscsi devstack@keystone.service[80346]: current working directory: / Jun 16 17:29:01.902826 dv-cinder-837542-4-iscsi devstack@keystone.service[80346]: detected binary path: /usr/bin/uwsgi-core Jun 16 17:29:01.902826 dv-cinder-837542-4-iscsi devstack@keystone.service[80346]: your processes number limit is 47680 Jun 16 17:29:01.903535 dv-cinder-837542-4-iscsi devstack@keystone.service[80346]: your memory page size is 4096 bytes Jun 16 17:29:01.903535 dv-cinder-837542-4-iscsi devstack@keystone.service[80346]: detected max file descriptor number: 2048 Jun 16 17:29:01.903535 dv-cinder-837542-4-iscsi devstack@keystone.service[80346]: lock engine: pthread robust mutexes Jun 16 17:29:01.903535 dv-cinder-837542-4-iscsi devstack@keystone.service[80346]: thunder lock: enabled Jun 16 17:29:01.903535 dv-cinder-837542-4-iscsi devstack@keystone.service[80346]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jun 16 17:29:01.903535 dv-cinder-837542-4-iscsi devstack@keystone.service[80346]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jun 16 17:29:01.949451 dv-cinder-837542-4-iscsi devstack@keystone.service[80346]: Python main interpreter initialized at 0x5562445d29c0 Jun 16 17:29:01.949451 dv-cinder-837542-4-iscsi devstack@keystone.service[80346]: python threads support enabled Jun 16 17:29:01.949451 dv-cinder-837542-4-iscsi devstack@keystone.service[80346]: your server socket listen backlog is limited to 100 connections Jun 16 17:29:01.949451 dv-cinder-837542-4-iscsi devstack@keystone.service[80346]: your mercy for graceful operations on workers is 90 seconds Jun 16 17:29:01.949834 dv-cinder-837542-4-iscsi devstack@keystone.service[80346]: mapped 403077 bytes (393 KB) for 2 cores Jun 16 17:29:01.949909 dv-cinder-837542-4-iscsi devstack@keystone.service[80346]: *** Operational MODE: preforking *** Jun 16 17:29:01.950169 dv-cinder-837542-4-iscsi devstack@keystone.service[80346]: *** uWSGI is running in multiple interpreter mode *** Jun 16 17:29:01.950169 dv-cinder-837542-4-iscsi devstack@keystone.service[80346]: spawned uWSGI master process (pid: 80346) Jun 16 17:29:01.950403 dv-cinder-837542-4-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jun 16 17:29:01.953086 dv-cinder-837542-4-iscsi devstack@keystone.service[80346]: spawned uWSGI worker 1 (pid: 80357, cores: 1) Jun 16 17:29:01.953086 dv-cinder-837542-4-iscsi devstack@keystone.service[80346]: spawned uWSGI worker 2 (pid: 80358, cores: 1) Jun 16 17:29:01.953086 dv-cinder-837542-4-iscsi devstack@keystone.service[80346]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jun 16 17:29:07.670576 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 16 17:29:07.670576 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: from cryptography.utils import int_from_bytes Jun 16 17:29:07.671363 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 16 17:29:07.671363 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: from cryptography.utils import int_from_bytes Jun 16 17:29:07.675424 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 16 17:29:07.675424 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: from cryptography.utils import int_from_bytes Jun 16 17:29:07.675977 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 16 17:29:07.675977 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: from cryptography.utils import int_from_bytes Jun 16 17:29:07.721176 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 16 17:29:07.721366 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 16 17:29:07.721554 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] command line args: [] {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 16 17:29:07.721841 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 16 17:29:07.722023 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] ================================================================================ {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 16 17:29:07.722282 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] admin_token = **** {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:29:07.722535 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] config_dir = [] {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:29:07.722729 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:29:07.722944 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] config_source = [] {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:29:07.723111 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] debug = True {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:29:07.723312 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:29:07.723685 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:29:07.723868 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:29:07.724125 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:29:07.724406 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:29:07.724638 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] list_limit = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:29:07.724822 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] log_config_append = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:29:07.725009 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:29:07.725252 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] log_dir = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:29:07.725369 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] log_file = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:29:07.725614 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:29:07.725845 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:29:07.726096 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:29:07.726349 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:29:07.726433 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 16 17:29:07.726532 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:29:07.726641 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 16 17:29:07.726730 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:29:07.726844 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] command line args: [] {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 16 17:29:07.726962 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:29:07.727128 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 16 17:29:07.727282 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:29:07.727360 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ================================================================================ {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 16 17:29:07.727436 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:29:07.727583 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] admin_token = **** {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:29:07.727680 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:29:07.727847 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] config_dir = [] {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:29:07.727936 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:29:07.728040 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:29:07.728124 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:29:07.728226 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] config_source = [] {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:29:07.728303 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:29:07.728437 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] debug = True {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:29:07.728580 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:29:07.728666 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:29:07.728865 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:29:07.728948 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:29:07.729046 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:29:07.729306 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:29:07.729410 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.42/identity {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:29:07.729555 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] publish_errors = False {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:29:07.729634 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:29:07.729788 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] list_limit = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:29:07.729864 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:29:07.729993 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] log_config_append = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:29:07.730070 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:29:07.730190 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:29:07.730271 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:29:07.730343 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] log_dir = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:29:07.730454 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:29:07.730537 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] log_file = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:29:07.730665 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:29:07.730749 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:29:07.730908 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] standard_threads = False {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:29:07.730984 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:29:07.731179 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:29:07.731276 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:29:07.731397 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:29:07.731481 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:29:07.731559 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:29:07.731634 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:29:07.731738 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:29:07.731817 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] use_journal = False {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:29:07.731913 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] use_json = False {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:29:07.731988 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:29:07.732092 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] use_stderr = False {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:29:07.732222 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:29:07.732307 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] use_syslog = False {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:29:07.732490 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:29:07.732570 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:29:07.732730 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:29:07.732835 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.732966 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:29:07.733044 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.733202 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:29:07.733283 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.733454 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:29:07.733531 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.733704 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:29:07.733784 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.733962 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.734044 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:29:07.734217 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.734461 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.734543 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.42/identity {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:29:07.734644 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] publish_errors = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:29:07.734719 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.734880 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:29:07.734964 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.735164 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:29:07.735375 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:29:07.735457 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.735724 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.735807 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:29:07.735986 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.736239 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.736330 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:29:07.736516 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] standard_threads = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:29:07.736594 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.736814 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:29:07.737012 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.737131 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:29:07.737257 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.737372 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:29:07.737497 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.737620 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] use_journal = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:29:07.737620 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] use_json = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:29:07.737833 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.737962 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] use_stderr = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:29:07.738070 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.738197 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] use_syslog = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:29:07.738323 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.738432 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:29:07.738554 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.738662 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.738662 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.738878 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.739000 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.739116 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.739241 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.739361 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.739479 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.739587 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.739790 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.739899 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.740011 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.740123 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.740235 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.740411 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.740550 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.740667 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.740775 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.740895 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.741012 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.741133 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.741254 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.741372 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.741564 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.741685 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.741800 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.741911 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.742035 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.742153 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.742260 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.742406 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.742512 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.742625 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.742753 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.742859 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.742970 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.743097 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.743204 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.743314 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.743437 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.743545 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.743545 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.743768 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.743876 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.743987 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.744110 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.744216 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.744328 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.744453 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.744646 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.744777 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.744887 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.744999 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.745109 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.745219 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.745330 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.745524 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.745631 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.745744 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.745855 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.745965 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.746076 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.746197 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.746318 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.746428 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.746551 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.746659 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.746820 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.746940 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.747056 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.747174 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.747281 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.747393 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.747503 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.747625 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.747741 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.747888 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.748040 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.748200 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.748362 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.748553 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.748701 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.748845 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.749018 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.749178 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.749369 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.749541 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.749715 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.749871 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.750056 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.750178 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.750387 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.750523 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.750523 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.750777 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.750898 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.751062 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.751186 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.751366 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.751539 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.751843 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.751975 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.752099 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.752261 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.752399 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.752587 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.752708 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.752870 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.752989 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.752989 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.753252 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.753391 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.753511 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.753689 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.753689 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.753939 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.754056 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.754229 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.754344 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.754505 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.754656 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.754820 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.754929 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.755090 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.755200 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.755316 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.755482 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.755600 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.755713 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.755825 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.755955 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.756063 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.756187 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.756308 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.756447 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.756581 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.756696 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.756808 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.756921 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.757033 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.757146 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.757272 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.757381 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.757542 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.757652 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.757776 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.757886 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.758023 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.758140 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.758252 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.758362 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.758477 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.758587 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.758787 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.758897 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.759012 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.759123 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.759234 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.759350 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.759482 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.759603 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.759725 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.759848 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.759958 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.760068 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] auth.external = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.760232 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.760350 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.760586 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.760697 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.760862 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.760972 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.761103 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.761223 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] auth.password = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.761345 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.761464 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] auth.token = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.761575 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.761940 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.762061 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.762171 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.762298 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.762407 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.762519 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.762630 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.762743 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.762743 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.762965 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.763080 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.763223 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.763333 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.763447 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.763569 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.763713 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] credential.caching = True {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.763834 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.763955 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.764065 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.764178 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.764290 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.764429 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.764552 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.764682 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.764793 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.764958 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.765070 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.765183 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.765312 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.765422 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.765533 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.765646 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.765758 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] auth.external = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.765871 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.765996 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.766107 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.766225 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.767153 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.767264 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.767385 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.767512 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] auth.password = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.767698 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.767824 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] auth.token = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.767938 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.768051 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.768167 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.768297 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.768498 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] federation.caching = True {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.768610 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.768725 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.768848 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.768997 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.769131 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.769245 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.769361 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.769478 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.769594 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.769594 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] credential.caching = True {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.769825 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.769825 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.770041 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.770156 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.770156 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.770372 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.770496 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.770496 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.770713 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.770828 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.770946 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.771064 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] identity.caching = True {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.771191 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.771305 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.771432 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.771545 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.771670 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.771670 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.771905 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.772019 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.772146 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.772260 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.772405 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.772405 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.772640 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.772754 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.772874 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.772993 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.773113 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.773232 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.773362 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.773479 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.773595 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.773713 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.773831 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] federation.caching = True {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.773950 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.774163 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.774277 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.774399 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.774517 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.774635 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.774754 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.774887 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.774887 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.775140 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.775268 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.775395 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.775531 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.775646 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.775766 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.775886 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.776015 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.776133 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.776270 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.776426 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.776541 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.776658 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.776775 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] identity.caching = True {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.776893 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.777031 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.777195 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.777328 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.777439 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.777552 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.777669 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.777804 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.777916 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.778032 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.778148 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.778264 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.778381 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.778509 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.778623 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.778743 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.778861 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.778977 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.779092 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.779217 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.779330 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.779447 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.779562 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.779677 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.779793 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.779921 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.780033 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.780149 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.780270 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.780406 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.780533 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.780670 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.780785 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.780903 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.781032 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.781032 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.781243 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.781243 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.781474 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.781593 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.781712 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.781832 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.781958 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.782639 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.782767 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.782884 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.783001 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.783111 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.783219 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.783339 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.783447 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.783555 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.783672 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.783789 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.783908 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.784045 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.784174 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] ldap.user = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.784301 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.784479 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.784592 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.784709 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.784842 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.784955 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.785072 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.785190 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.785307 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.785422 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.785550 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.785664 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.785785 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.785901 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.786017 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.786134 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.786134 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.786361 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.786477 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.786477 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.786691 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.786691 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.786921 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.787033 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.787153 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.787266 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.787376 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.787487 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.787610 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.787734 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.787852 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.787968 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.788085 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.788274 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.788404 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.788505 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.788613 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.788721 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.788836 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.788977 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.789091 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.789210 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.789328 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.789441 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.789551 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.789714 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.user = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.789830 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.789949 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.790067 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.790185 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.790301 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.790432 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.790546 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.790671 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.790788 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.790906 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.791016 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.791135 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.791245 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.791366 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.791495 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.791612 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.791731 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.791861 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.791980 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.792099 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.792220 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.792344 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.792542 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.792678 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.792793 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.792914 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.793035 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] resource.caching = True {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.793166 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.793283 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.793416 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.793533 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.793655 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.793776 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.793895 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.794020 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.794152 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.794269 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.794390 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.794510 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.794630 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.794750 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.794880 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.794997 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.795117 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.795235 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.795354 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.795472 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] role.caching = True {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.795602 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.795725 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] role.driver = sql {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.795844 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.795964 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.796085 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.796204 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.796334 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.796520 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.796631 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.796753 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.796871 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.796993 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.797129 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.797244 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.797364 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.797489 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.797671 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.797792 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.797997 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.798109 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.798228 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.798348 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.798467 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.798586 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.798715 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.798715 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] resource.caching = True {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.798923 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.798923 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.799131 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.799248 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.799381 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.799381 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.799591 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.799707 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.799825 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.799962 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.800077 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.800197 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.800318 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.800461 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.800573 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.800705 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.800705 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.800903 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.800903 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.801102 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] role.caching = True {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.801210 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.801336 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] role.driver = sql {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.801336 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.801597 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.801709 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.801862 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.801862 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.802086 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.802198 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.802320 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.802435 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.802555 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.802670 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.802798 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.802911 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.803036 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.803150 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.803268 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.803385 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.803513 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.803627 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.803744 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.803865 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.803981 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.804096 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] token.caching = True {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.804223 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.804336 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.804518 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.804635 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.804759 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.804881 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.804881 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.805118 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.805238 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.805357 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.805478 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.805602 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.805737 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.805856 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.805976 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.806097 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.806216 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.806335 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.806467 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.806586 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.806708 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.806827 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.806951 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.807069 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.807205 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.807322 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.807443 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.807562 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.807681 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.807811 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.807928 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.808049 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.808049 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.808262 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.808415 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.808558 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] database.connection = **** {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.808681 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.808813 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.808931 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.809047 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.809164 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.809280 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.809398 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.809526 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.809639 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] token.caching = True {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.809758 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.809875 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.809991 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.810106 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.810235 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.810348 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.810464 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.810585 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.810701 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.810819 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.810949 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.811062 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.811178 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.811295 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.811411 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.811526 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.811654 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.811768 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.811885 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.812003 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.812123 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.812240 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.812386 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.812509 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.812630 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.812747 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.812864 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.812864 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80357) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 16 17:29:07.813104 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.813104 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.813104 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.813104 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.813104 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] database.connection = **** {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.813104 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.813104 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.813104 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.813104 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.814067 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.814067 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.814067 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.814067 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.814067 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.814067 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.814067 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.814067 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.814067 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.815034 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.815034 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.815034 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.815369 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.815735 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:29:07.816043 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 16 17:29:07.841887 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80357) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:29:07.842131 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80357) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:29:07.842335 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80357) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:29:07.934545 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:29:07.934724 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:29:07.934931 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:29:07.938535 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80357) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:29:07.943977 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:29:07.968254 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80357) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:29:07.968513 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80357) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:29:07.973025 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:29:07.973262 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:29:08.023451 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80357) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:29:08.024298 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:29:08.074920 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80357) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:29:08.075164 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:29:08.075252 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80357) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:29:08.075316 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:29:08.144406 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80357) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:29:08.144650 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:29:08.144650 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:29:08.144899 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80357) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:29:08.144990 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:29:08.145067 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:29:08.145181 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80357) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:29:08.145316 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:29:08.145480 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:29:08.145625 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80357) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:29:08.145962 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80357) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:29:08.146341 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80357) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:29:08.169728 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:29:08.169979 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:29:08.170216 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:29:08.170363 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80357) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:29:08.170461 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:29:08.170602 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80357) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:29:08.170705 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:29:08.170828 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80357) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:29:08.171053 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80357) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:29:08.171149 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:29:08.171266 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80357) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:29:08.171384 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:29:08.171513 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80357) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:29:08.171686 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80357) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:29:08.183912 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80357) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:29:08.187547 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:29:08.198985 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80357) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:29:08.199260 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80357) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:29:08.199424 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80357) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:29:08.199651 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80357) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:29:08.199874 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80357) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:29:08.200093 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80357) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:29:08.200305 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80357) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:29:08.200429 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:29:08.200636 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:29:08.200860 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:29:08.201088 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:29:08.201303 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:29:08.201521 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:29:08.201736 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:29:08.218826 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.219075 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 16 17:29:08.219152 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.219262 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 16 17:29:08.219752 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.220103 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 16 17:29:08.220645 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.220947 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.221219 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.221482 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.221703 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.221919 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 16 17:29:08.222219 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.222495 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.222763 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.223032 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.232584 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.232874 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.233144 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.233423 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.234182 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.234472 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.234744 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.235061 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.242337 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80358) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:29:08.243411 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80357) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:29:08.245687 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80358) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:29:08.245965 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.246247 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.246522 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.246802 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.246887 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80357) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:29:08.247060 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.247149 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.247327 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.247413 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.247599 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.247709 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.247867 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.247949 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.248138 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.248228 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.248421 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.248506 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.248743 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.249015 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.249293 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.249559 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.277576 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.277808 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.279235 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80358) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:29:08.279334 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80357) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:29:08.279516 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.279601 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.285069 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80357) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:29:08.285290 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80358) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:29:08.285379 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.285624 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.285706 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.285984 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.293204 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80357) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:29:08.293500 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.293649 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80358) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:29:08.293772 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.298370 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80357) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:29:08.298619 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80358) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:29:08.298735 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.298938 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.299063 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.299227 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.299361 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.299489 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.299620 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.299776 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.299902 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.300017 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.300145 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.300268 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.300364 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.300554 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.319798 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.320087 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.320258 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.320579 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.324742 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80357) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:29:08.325210 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80358) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:29:08.328477 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.328802 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.329553 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.330004 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.334748 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80357) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:29:08.335402 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80358) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:29:08.338425 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80357) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:29:08.339062 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80358) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:29:08.342087 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.342394 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.342679 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.342853 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.342975 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.343160 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.343282 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.343444 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.343562 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.343730 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.344004 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.344280 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.363376 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80357) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:29:08.363673 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.363949 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.364211 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.364408 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80358) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:29:08.364712 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.364980 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.365247 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.372252 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.373535 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.374023 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.374315 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.375286 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.375583 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.378118 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80357) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:29:08.378895 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.378895 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.379239 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.379612 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.381096 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80358) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:29:08.381398 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.381680 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.381969 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.382249 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.394400 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80357) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:29:08.394779 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.395142 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.395485 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.395830 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.396183 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.396565 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.397178 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80358) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:29:08.397615 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.397930 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.398211 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.398594 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.398868 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.399145 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.415313 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80357) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:29:08.417552 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80358) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:29:08.418632 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80357) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:29:08.420835 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80358) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:29:08.421982 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.424216 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.425008 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.426022 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80357) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:29:08.426304 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.426588 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.426751 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.428412 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80358) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:29:08.428694 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.428973 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.433931 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.435573 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80357) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:29:08.436278 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.438012 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80358) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:29:08.439525 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.439830 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.440110 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.440412 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.442699 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.442999 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.443415 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.444073 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.449987 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80357) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:29:08.450294 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.450580 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.453027 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80358) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:29:08.453324 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.453604 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.458740 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80357) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:29:08.459029 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.459315 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.459593 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.459868 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.460150 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.460452 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.460737 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.461013 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.461398 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.461676 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.461944 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.462215 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.462327 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80358) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:29:08.462497 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80357) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.462622 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.462902 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.463178 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.463441 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.463857 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.464133 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.464430 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.464710 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.464992 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.465287 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.465521 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.465797 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.466057 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:29:08.502041 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x5562445d29c0 pid: 80357 (default app) Jun 16 17:29:08.506202 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d0715e-f15b-4681-b216-f7efe6d380b5 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:08.506570 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d0715e-f15b-4681-b216-f7efe6d380b5 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:08.506780 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x5562445d29c0 pid: 80358 (default app) Jun 16 17:29:08.506947 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d0715e-f15b-4681-b216-f7efe6d380b5 None None] PATH_INFO: `/v3` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:08.511370 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1/1] 10.222.0.42 () {50 vars in 791 bytes} [Thu Jun 16 17:29:08 2022] GET /identity/v3/ => generated 255 bytes in 9 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:29:16.761184 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a042073-b31f-453f-b974-a04e91995168 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:16.762194 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a042073-b31f-453f-b974-a04e91995168 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:16.762194 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a042073-b31f-453f-b974-a04e91995168 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:16.767437 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1/2] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:29:16 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:29:16.773112 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779310d3-d0a0-4979-abc5-1cc2d4161211 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:16.773627 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779310d3-d0a0-4979-abc5-1cc2d4161211 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:16.773721 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779310d3-d0a0-4979-abc5-1cc2d4161211 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:16.843219 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: INFO dbcounter [None req-779310d3-d0a0-4979-abc5-1cc2d4161211 None None] Registered counter for database keystone Jun 16 17:29:16.865963 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG oslo_db.sqlalchemy.engines [None req-779310d3-d0a0-4979-abc5-1cc2d4161211 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80357) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 16 17:29:16.872431 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writer thread running {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 16 17:29:17.055769 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG passlib.handlers.bcrypt [None req-779310d3-d0a0-4979-abc5-1cc2d4161211 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80357) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 16 17:29:17.056079 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG passlib.handlers.bcrypt [None req-779310d3-d0a0-4979-abc5-1cc2d4161211 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80357) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 16 17:29:17.112082 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-779310d3-d0a0-4979-abc5-1cc2d4161211 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:29:17.113722 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-779310d3-d0a0-4979-abc5-1cc2d4161211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:29:17.243698 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 2/3] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:29:16 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 472 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 16 17:29:17.411812 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a042073-b31f-453f-b974-a04e91995168 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:17.482632 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: INFO dbcounter [None req-7a042073-b31f-453f-b974-a04e91995168 None None] Registered counter for database keystone Jun 16 17:29:17.503868 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG oslo_db.sqlalchemy.engines [None req-7a042073-b31f-453f-b974-a04e91995168 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80358) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 16 17:29:17.510388 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writer thread running {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 16 17:29:17.630979 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a042073-b31f-453f-b974-a04e91995168 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:17.635313 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7c9b4f-ec5c-435b-9e13-4bf90c43819a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:17.635988 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7c9b4f-ec5c-435b-9e13-4bf90c43819a admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:17.636272 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7c9b4f-ec5c-435b-9e13-4bf90c43819a admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:17.636578 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7c9b4f-ec5c-435b-9e13-4bf90c43819a admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:17.646252 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7c9b4f-ec5c-435b-9e13-4bf90c43819a admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:17.698202 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: WARNING keystone.common.rbac_enforcer.enforcer [None req-2f7c9b4f-ec5c-435b-9e13-4bf90c43819a admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 16 17:29:17.698454 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7c9b4f-ec5c-435b-9e13-4bf90c43819a admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:17.711785 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 2/4] 10.222.0.42 () {58 vars in 1324 bytes} [Thu Jun 16 17:29:17 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 17:29:17.717683 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779310d3-d0a0-4979-abc5-1cc2d4161211 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:17.733209 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779310d3-d0a0-4979-abc5-1cc2d4161211 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:17.737593 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f78cd3-f314-4150-8692-8889196b621f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:17.738712 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f78cd3-f314-4150-8692-8889196b621f admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:17.738994 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f78cd3-f314-4150-8692-8889196b621f admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:17.739273 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f78cd3-f314-4150-8692-8889196b621f admin admin] PATH_INFO: `/v3/projects/f64791acc5674278bd1baac8430d9719` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:17.740880 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f78cd3-f314-4150-8692-8889196b621f admin admin] RBAC: Authorizing `identity:get_project(project_id=f64791acc5674278bd1baac8430d9719)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:17.795242 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: WARNING keystone.common.rbac_enforcer.enforcer [None req-f7f78cd3-f314-4150-8692-8889196b621f admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 16 17:29:17.795904 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f78cd3-f314-4150-8692-8889196b621f admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:17.800298 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 3/5] 10.222.0.42 () {56 vars in 1207 bytes} [Thu Jun 16 17:29:17 2022] GET /identity/v3/projects/f64791acc5674278bd1baac8430d9719 => generated 344 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:29:19.929180 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7cde2e-3239-4510-ade8-273392b42d2d None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:19.929681 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7cde2e-3239-4510-ade8-273392b42d2d None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:19.929799 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7cde2e-3239-4510-ade8-273392b42d2d None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:19.933871 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 3/6] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:29:19 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:29:19.940050 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21506547-fa09-4709-a643-e2b7dafc0cb9 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:19.940492 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21506547-fa09-4709-a643-e2b7dafc0cb9 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:19.940618 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21506547-fa09-4709-a643-e2b7dafc0cb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:19.959691 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-21506547-fa09-4709-a643-e2b7dafc0cb9 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:29:19.961227 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-21506547-fa09-4709-a643-e2b7dafc0cb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:29:19.968997 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 4/7] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:29:19 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 16 17:29:19.991706 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7cde2e-3239-4510-ade8-273392b42d2d None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:20.004802 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7cde2e-3239-4510-ade8-273392b42d2d None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:20.009153 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7c5045-9370-4745-9af8-6e2eebddfaeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:20.009797 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7c5045-9370-4745-9af8-6e2eebddfaeb admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:20.010085 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7c5045-9370-4745-9af8-6e2eebddfaeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:20.010362 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7c5045-9370-4745-9af8-6e2eebddfaeb admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:20.019334 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7c5045-9370-4745-9af8-6e2eebddfaeb admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:20.021255 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7c5045-9370-4745-9af8-6e2eebddfaeb admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:20.029995 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 4/8] 10.222.0.42 () {58 vars in 1324 bytes} [Thu Jun 16 17:29:19 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 16 17:29:20.036658 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21506547-fa09-4709-a643-e2b7dafc0cb9 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:20.045843 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21506547-fa09-4709-a643-e2b7dafc0cb9 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:20.051189 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44507230-f297-48c0-bd5f-15d6b0b858f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:20.052071 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44507230-f297-48c0-bd5f-15d6b0b858f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:20.052360 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44507230-f297-48c0-bd5f-15d6b0b858f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:20.052671 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44507230-f297-48c0-bd5f-15d6b0b858f3 admin admin] PATH_INFO: `/v3/users/12a220316b144048ad50f0affd87fa43` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:20.054086 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44507230-f297-48c0-bd5f-15d6b0b858f3 admin admin] RBAC: Authorizing `identity:get_user(user_id=12a220316b144048ad50f0affd87fa43)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:20.055945 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44507230-f297-48c0-bd5f-15d6b0b858f3 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:20.057607 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 5/9] 10.222.0.42 () {56 vars in 1198 bytes} [Thu Jun 16 17:29:20 2022] GET /identity/v3/users/12a220316b144048ad50f0affd87fa43 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:29:22.306790 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf00cda-c8b4-47bd-8cff-420b368608fb None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:22.307313 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf00cda-c8b4-47bd-8cff-420b368608fb None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:22.307395 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf00cda-c8b4-47bd-8cff-420b368608fb None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:22.308763 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 5/10] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:29:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:29:22.314505 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e3cd44-cb1f-44b3-acd8-5358b1cf3787 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:22.314968 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e3cd44-cb1f-44b3-acd8-5358b1cf3787 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:22.315064 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e3cd44-cb1f-44b3-acd8-5358b1cf3787 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:22.320803 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d7069e-7a80-42ab-917b-d2f2358803d8 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:22.321100 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d7069e-7a80-42ab-917b-d2f2358803d8 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:22.321385 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d7069e-7a80-42ab-917b-d2f2358803d8 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:22.322730 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 6/11] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:29:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:29:22.327726 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615c087b-dfaf-4a50-aff1-f222e22265c3 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:22.328024 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615c087b-dfaf-4a50-aff1-f222e22265c3 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:22.328304 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615c087b-dfaf-4a50-aff1-f222e22265c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:22.332342 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-c3e3cd44-cb1f-44b3-acd8-5358b1cf3787 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:29:22.333572 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-c3e3cd44-cb1f-44b3-acd8-5358b1cf3787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:29:22.369317 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG passlib.handlers.bcrypt [None req-615c087b-dfaf-4a50-aff1-f222e22265c3 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80358) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 16 17:29:22.369743 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG passlib.handlers.bcrypt [None req-615c087b-dfaf-4a50-aff1-f222e22265c3 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80358) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 16 17:29:22.415817 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 6/12] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:29:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:29:22.416020 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-615c087b-dfaf-4a50-aff1-f222e22265c3 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:29:22.417527 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-615c087b-dfaf-4a50-aff1-f222e22265c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:29:22.439169 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e3cd44-cb1f-44b3-acd8-5358b1cf3787 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:22.464781 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:22.519363 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 7/13] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:29:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 17:29:22.542730 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615c087b-dfaf-4a50-aff1-f222e22265c3 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:22.549990 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e3cd44-cb1f-44b3-acd8-5358b1cf3787 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:22.584694 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ca7b5e-92d3-4373-a9a0-2e9e76c9bf29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:22.585322 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ca7b5e-92d3-4373-a9a0-2e9e76c9bf29 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:22.585605 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ca7b5e-92d3-4373-a9a0-2e9e76c9bf29 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:22.585886 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ca7b5e-92d3-4373-a9a0-2e9e76c9bf29 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:22.644533 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615c087b-dfaf-4a50-aff1-f222e22265c3 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:22.646855 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:22.658646 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ca7b5e-92d3-4373-a9a0-2e9e76c9bf29 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:22.660121 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ca7b5e-92d3-4373-a9a0-2e9e76c9bf29 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:22.683544 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47515063-9826-4e42-a753-daee04467336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:22.684511 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:22.684617 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47515063-9826-4e42-a753-daee04467336 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:22.684891 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47515063-9826-4e42-a753-daee04467336 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:22.685169 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47515063-9826-4e42-a753-daee04467336 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:22.698584 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47515063-9826-4e42-a753-daee04467336 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:29:22.710135 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-47515063-9826-4e42-a753-daee04467336 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:29:22.710135 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:29:22.710135 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:29:22.710135 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:29:22.710135 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:29:22.710135 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:29:22.710135 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:29:22.710135 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:29:22.710135 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:29:22.710135 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:29:22.710135 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:29:22.710135 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:29:22.710135 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:29:22.710135 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:29:22.710135 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:29:22.710135 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:29:22.711077 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:29:22.711077 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:29:22.711077 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:29:22.711077 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:29:22.711077 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:29:22.711077 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:29:22.711077 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:29:22.711077 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:29:22.711077 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:29:22.711077 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:29:22.711077 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:29:22.711077 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:29:22.711077 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:29:22.711077 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:29:22.711077 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:29:22.711077 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:29:22.711864 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:29:22.711864 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:29:22.711864 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:29:22.711864 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:29:22.711864 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:29:22.711864 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:29:22.711864 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:29:22.711864 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:29:22.711864 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:29:22.711864 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:29:22.711864 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:29:22.711864 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:29:22.711864 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 8/14] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Jun 16 17:29:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:29:22.717089 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47515063-9826-4e42-a753-daee04467336 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:22.764539 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 7/15] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jun 16 17:29:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 17:29:22.770736 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ca7b5e-92d3-4373-a9a0-2e9e76c9bf29 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:22.820876 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47515063-9826-4e42-a753-daee04467336 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:22.858664 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca85f1d-791d-4552-bbed-6b6a0d29a768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:22.859233 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca85f1d-791d-4552-bbed-6b6a0d29a768 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:22.859528 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca85f1d-791d-4552-bbed-6b6a0d29a768 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:22.859806 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca85f1d-791d-4552-bbed-6b6a0d29a768 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:22.861097 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca85f1d-791d-4552-bbed-6b6a0d29a768 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:29:22.861407 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca85f1d-791d-4552-bbed-6b6a0d29a768 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:22.862889 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca85f1d-791d-4552-bbed-6b6a0d29a768 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:22.873583 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ca7b5e-92d3-4373-a9a0-2e9e76c9bf29 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:22.877860 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 9/16] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jun 16 17:29:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:29:22.883584 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca85f1d-791d-4552-bbed-6b6a0d29a768 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:22.904223 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:22.913312 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:22.916960 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bcff9e-a83b-447f-b6bb-666c87286c23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:22.917824 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bcff9e-a83b-447f-b6bb-666c87286c23 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:22.918046 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bcff9e-a83b-447f-b6bb-666c87286c23 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:22.918329 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bcff9e-a83b-447f-b6bb-666c87286c23 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:22.931799 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bcff9e-a83b-447f-b6bb-666c87286c23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:22.933312 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bcff9e-a83b-447f-b6bb-666c87286c23 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:22.955769 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-a7bcff9e-a83b-447f-b6bb-666c87286c23 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:29:22.955769 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:29:22.955769 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:29:22.955769 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:29:22.955769 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:29:22.955769 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:29:22.955769 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:29:22.955769 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:29:22.955769 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:29:22.955769 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:29:22.955769 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:29:22.955769 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:29:22.955769 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:29:22.955769 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:29:22.955769 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:29:22.956715 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:29:22.956715 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:29:22.956715 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:29:22.956715 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:29:22.956715 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:29:22.956715 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:29:22.956715 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:29:22.956715 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:29:22.956715 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:29:22.956715 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:29:22.956715 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:29:22.956715 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:29:22.956715 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:29:22.956715 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:29:22.956715 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:29:22.956715 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:29:22.956715 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:29:22.957679 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:29:22.957679 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:29:22.957679 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:29:22.957679 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:29:22.957679 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 8/17] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jun 16 17:29:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:29:22.962927 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bcff9e-a83b-447f-b6bb-666c87286c23 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:22.996391 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca85f1d-791d-4552-bbed-6b6a0d29a768 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:23.038274 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0932dd9c-0231-49c6-899e-7bc3295928fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:23.039147 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0932dd9c-0231-49c6-899e-7bc3295928fe None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:23.039426 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0932dd9c-0231-49c6-899e-7bc3295928fe None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:23.039703 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0932dd9c-0231-49c6-899e-7bc3295928fe None admin] PATH_INFO: `/v3/users/12a220316b144048ad50f0affd87fa43` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:23.041306 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0932dd9c-0231-49c6-899e-7bc3295928fe None admin] RBAC: Authorizing `identity:get_user(user_id=12a220316b144048ad50f0affd87fa43)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:23.042761 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0932dd9c-0231-49c6-899e-7bc3295928fe None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:23.044416 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 10/18] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jun 16 17:29:22 2022] GET /identity/v3/users/12a220316b144048ad50f0affd87fa43 => generated 247 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:29:23.050684 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0932dd9c-0231-49c6-899e-7bc3295928fe None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:23.081988 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bcff9e-a83b-447f-b6bb-666c87286c23 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:23.118193 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25de014f-1baa-46c4-8a87-87e4a3f01e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:23.118786 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25de014f-1baa-46c4-8a87-87e4a3f01e7b None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:23.119063 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25de014f-1baa-46c4-8a87-87e4a3f01e7b None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:23.119332 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25de014f-1baa-46c4-8a87-87e4a3f01e7b None admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:23.120053 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25de014f-1baa-46c4-8a87-87e4a3f01e7b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:29:23.120362 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25de014f-1baa-46c4-8a87-87e4a3f01e7b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:23.121771 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25de014f-1baa-46c4-8a87-87e4a3f01e7b None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:23.139507 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 9/19] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jun 16 17:29:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:29:23.147957 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:23.156665 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0932dd9c-0231-49c6-899e-7bc3295928fe None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:23.197437 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628f87b6-e32b-440d-b2ca-ece51f86e633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:23.198269 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628f87b6-e32b-440d-b2ca-ece51f86e633 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:23.198563 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628f87b6-e32b-440d-b2ca-ece51f86e633 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:23.198838 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628f87b6-e32b-440d-b2ca-ece51f86e633 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:23.200169 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628f87b6-e32b-440d-b2ca-ece51f86e633 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:23.201728 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628f87b6-e32b-440d-b2ca-ece51f86e633 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:23.203487 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 11/20] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jun 16 17:29:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:29:23.209720 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25de014f-1baa-46c4-8a87-87e4a3f01e7b None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:23.249080 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:23.323954 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25de014f-1baa-46c4-8a87-87e4a3f01e7b None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:23.360976 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf7aa63-5add-491e-b434-b8f3ccea90a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:23.361644 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf7aa63-5add-491e-b434-b8f3ccea90a0 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:23.361932 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf7aa63-5add-491e-b434-b8f3ccea90a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:23.362216 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf7aa63-5add-491e-b434-b8f3ccea90a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:23.363056 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf7aa63-5add-491e-b434-b8f3ccea90a0 None admin] RBAC: Adding query filter params (role.id=3ca2dcbbd29d4a31a3ab78f2299bc239, scope.domain.id=default, user.id=12a220316b144048ad50f0affd87fa43) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:29:23.363372 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf7aa63-5add-491e-b434-b8f3ccea90a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:23.364878 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf7aa63-5add-491e-b434-b8f3ccea90a0 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:23.372332 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 10/21] 10.222.0.42 () {56 vars in 1322 bytes} [Thu Jun 16 17:29:23 2022] GET /identity/v3/role_assignments?user.id=12a220316b144048ad50f0affd87fa43&scope.domain.id=default&role.id=3ca2dcbbd29d4a31a3ab78f2299bc239 => generated 233 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:29:25.898941 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c4ab3e-e9f4-45a0-9795-9df0384a0566 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:25.899439 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c4ab3e-e9f4-45a0-9795-9df0384a0566 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:25.899928 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c4ab3e-e9f4-45a0-9795-9df0384a0566 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:25.901655 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 12/22] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:29:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:29:25.908477 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78687e8f-deb4-4d8c-b085-e091951c9509 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:25.909017 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78687e8f-deb4-4d8c-b085-e091951c9509 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:25.909397 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78687e8f-deb4-4d8c-b085-e091951c9509 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:25.941581 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-78687e8f-deb4-4d8c-b085-e091951c9509 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:29:25.942837 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-78687e8f-deb4-4d8c-b085-e091951c9509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:29:25.990940 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e086855-c34b-42b1-92a6-6150f08e48e1 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:25.991869 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e086855-c34b-42b1-92a6-6150f08e48e1 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:25.992577 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e086855-c34b-42b1-92a6-6150f08e48e1 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:25.994491 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 13/23] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:29:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:29:26.002249 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10873680-4a68-4461-8cce-b0629edea0f0 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:26.002963 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10873680-4a68-4461-8cce-b0629edea0f0 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:26.003501 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10873680-4a68-4461-8cce-b0629edea0f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:26.036982 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-10873680-4a68-4461-8cce-b0629edea0f0 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:29:26.038749 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-10873680-4a68-4461-8cce-b0629edea0f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:29:26.082509 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 11/24] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:29:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:29:26.091933 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20094e7e-4a6a-4813-b50a-a60e38925e4c None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:26.092492 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20094e7e-4a6a-4813-b50a-a60e38925e4c None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:26.092916 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20094e7e-4a6a-4813-b50a-a60e38925e4c None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:26.094530 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 12/25] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:29:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:29:26.101999 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82412b25-c2de-47b8-b3b4-9811b1e55a06 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:26.102730 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82412b25-c2de-47b8-b3b4-9811b1e55a06 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:26.103154 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82412b25-c2de-47b8-b3b4-9811b1e55a06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:26.141830 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-82412b25-c2de-47b8-b3b4-9811b1e55a06 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:29:26.143855 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-82412b25-c2de-47b8-b3b4-9811b1e55a06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:29:26.176487 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 14/26] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:29:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 17:29:26.177681 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10873680-4a68-4461-8cce-b0629edea0f0 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:26.206727 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:26.303351 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 13/27] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:29:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:29:26.304774 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82412b25-c2de-47b8-b3b4-9811b1e55a06 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:26.364652 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:26.376458 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10873680-4a68-4461-8cce-b0629edea0f0 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:26.417675 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a929e230-112b-4dfa-b6d8-efb44fb6f7bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:26.418757 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a929e230-112b-4dfa-b6d8-efb44fb6f7bf None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:26.419142 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a929e230-112b-4dfa-b6d8-efb44fb6f7bf None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:26.419648 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a929e230-112b-4dfa-b6d8-efb44fb6f7bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:26.424930 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82412b25-c2de-47b8-b3b4-9811b1e55a06 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:26.440838 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a929e230-112b-4dfa-b6d8-efb44fb6f7bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:26.443024 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a929e230-112b-4dfa-b6d8-efb44fb6f7bf None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:26.464386 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-a929e230-112b-4dfa-b6d8-efb44fb6f7bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:29:26.464386 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:29:26.464386 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:29:26.464386 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:29:26.464386 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:29:26.464386 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:29:26.464386 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:29:26.464386 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:29:26.464386 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:29:26.464386 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:29:26.464386 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:29:26.464386 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:29:26.464386 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:29:26.464386 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:29:26.464386 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:29:26.465249 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:29:26.465249 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:29:26.465249 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:29:26.465249 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:29:26.465249 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:29:26.465249 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:29:26.465249 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:29:26.465249 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:29:26.465249 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:29:26.465249 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:29:26.465249 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:29:26.465249 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:29:26.465249 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:29:26.465249 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:29:26.465249 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:29:26.465249 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:29:26.465249 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:29:26.466041 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:29:26.466041 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:29:26.466041 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:29:26.466041 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:29:26.467965 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 15/28] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jun 16 17:29:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:29:26.474002 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f5e10a-9f23-4c0e-ab82-33716d061bff None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:26.474002 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f5e10a-9f23-4c0e-ab82-33716d061bff None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:26.474002 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f5e10a-9f23-4c0e-ab82-33716d061bff None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:26.475189 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 16/29] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:29:26 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:29:26.476924 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f5e10a-9f23-4c0e-ab82-33716d061bff None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:26.480495 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415a0db6-d1cd-461a-826e-82930a3c4c00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:26.481307 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415a0db6-d1cd-461a-826e-82930a3c4c00 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:26.481593 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415a0db6-d1cd-461a-826e-82930a3c4c00 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:26.481872 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415a0db6-d1cd-461a-826e-82930a3c4c00 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:26.496863 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415a0db6-d1cd-461a-826e-82930a3c4c00 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:29:26.507123 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-415a0db6-d1cd-461a-826e-82930a3c4c00 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:29:26.507123 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:29:26.507123 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:29:26.507123 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:29:26.507123 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:29:26.507123 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:29:26.507123 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:29:26.507123 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:29:26.507123 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:29:26.507123 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:29:26.507123 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:29:26.507123 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:29:26.507123 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:29:26.507123 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:29:26.507123 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:29:26.507123 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:29:26.508007 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:29:26.508007 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:29:26.508007 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:29:26.508007 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:29:26.508007 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:29:26.508007 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:29:26.508007 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:29:26.508007 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:29:26.508007 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:29:26.508007 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:29:26.508007 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:29:26.508007 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:29:26.508007 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:29:26.508007 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:29:26.508007 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:29:26.508007 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:29:26.508839 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:29:26.508839 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:29:26.508839 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:29:26.508839 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:29:26.508839 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:29:26.508839 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:29:26.508839 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:29:26.508839 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:29:26.508839 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:29:26.508839 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:29:26.508839 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:29:26.508839 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:29:26.510850 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 14/30] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Jun 16 17:29:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:29:26.511819 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415a0db6-d1cd-461a-826e-82930a3c4c00 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:26.540768 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:26.615593 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f5e10a-9f23-4c0e-ab82-33716d061bff None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:26.635041 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415a0db6-d1cd-461a-826e-82930a3c4c00 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:26.664319 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cc4fa4-8e69-463d-bd45-d04a9ab649d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:26.665029 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cc4fa4-8e69-463d-bd45-d04a9ab649d9 None admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:26.665320 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cc4fa4-8e69-463d-bd45-d04a9ab649d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:26.665843 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cc4fa4-8e69-463d-bd45-d04a9ab649d9 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:26.666672 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cc4fa4-8e69-463d-bd45-d04a9ab649d9 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:26.668838 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cc4fa4-8e69-463d-bd45-d04a9ab649d9 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:26.675259 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5d0848-d129-4275-b238-827813dc03da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:26.676430 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5d0848-d129-4275-b238-827813dc03da None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:26.677059 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5d0848-d129-4275-b238-827813dc03da None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:26.677606 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5d0848-d129-4275-b238-827813dc03da None admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:26.678432 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5d0848-d129-4275-b238-827813dc03da None admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:29:26.678891 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5d0848-d129-4275-b238-827813dc03da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:26.680406 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5d0848-d129-4275-b238-827813dc03da None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:26.699530 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 15/31] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jun 16 17:29:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:29:26.701059 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd09aa08-c82b-4651-ad75-0e7d3c8d6a95 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:26.701354 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd09aa08-c82b-4651-ad75-0e7d3c8d6a95 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:26.701637 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd09aa08-c82b-4651-ad75-0e7d3c8d6a95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:26.721864 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-cd09aa08-c82b-4651-ad75-0e7d3c8d6a95 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:29:26.723131 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-cd09aa08-c82b-4651-ad75-0e7d3c8d6a95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:29:26.808714 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 17/32] 10.222.0.42 () {60 vars in 1131 bytes} [Thu Jun 16 17:29:26 2022] POST /identity/v3/roles => generated 219 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:29:26.809842 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cc4fa4-8e69-463d-bd45-d04a9ab649d9 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:26.812275 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 16/33] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:29:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:29:26.815225 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd09aa08-c82b-4651-ad75-0e7d3c8d6a95 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:26.816870 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:26.835477 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:26.956932 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cc4fa4-8e69-463d-bd45-d04a9ab649d9 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:26.970909 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd09aa08-c82b-4651-ad75-0e7d3c8d6a95 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:26.995612 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b43b0b-20ed-4b0d-95fe-101b67e758ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:26.996335 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b43b0b-20ed-4b0d-95fe-101b67e758ca None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:26.996648 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b43b0b-20ed-4b0d-95fe-101b67e758ca None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:26.996928 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b43b0b-20ed-4b0d-95fe-101b67e758ca None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:26.997697 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b43b0b-20ed-4b0d-95fe-101b67e758ca None admin] RBAC: Adding query filter params (name=admin) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:29:26.998006 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b43b0b-20ed-4b0d-95fe-101b67e758ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:26.999545 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b43b0b-20ed-4b0d-95fe-101b67e758ca None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:27.011253 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 18/34] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jun 16 17:29:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:29:27.012119 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b43b0b-20ed-4b0d-95fe-101b67e758ca None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:27.034121 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa2183c-391f-46a9-ad1e-513f94006e2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:27.034818 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa2183c-391f-46a9-ad1e-513f94006e2a None admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:27.035098 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa2183c-391f-46a9-ad1e-513f94006e2a None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:27.035371 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa2183c-391f-46a9-ad1e-513f94006e2a None admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:27.036101 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa2183c-391f-46a9-ad1e-513f94006e2a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:27.037658 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa2183c-391f-46a9-ad1e-513f94006e2a None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:27.149557 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:27.187661 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b43b0b-20ed-4b0d-95fe-101b67e758ca None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:27.336507 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 17/35] 10.222.0.42 () {60 vars in 1141 bytes} [Thu Jun 16 17:29:26 2022] POST /identity/v3/projects => generated 301 bytes in 523 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:29:27.337512 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa2183c-391f-46a9-ad1e-513f94006e2a None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:27.350945 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990a0caa-3c88-41ce-8e13-94423a29cdc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:27.352867 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990a0caa-3c88-41ce-8e13-94423a29cdc9 None admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:27.353492 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990a0caa-3c88-41ce-8e13-94423a29cdc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:27.354000 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990a0caa-3c88-41ce-8e13-94423a29cdc9 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:27.355064 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990a0caa-3c88-41ce-8e13-94423a29cdc9 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:27.357393 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990a0caa-3c88-41ce-8e13-94423a29cdc9 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:27.382104 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 19/36] 10.222.0.42 () {60 vars in 1131 bytes} [Thu Jun 16 17:29:27 2022] POST /identity/v3/roles => generated 225 bytes in 370 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:29:27.436737 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:27.477105 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ac9c27-c52a-4140-a94c-9f943e50fe2b None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:27.477740 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ac9c27-c52a-4140-a94c-9f943e50fe2b None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:27.478412 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ac9c27-c52a-4140-a94c-9f943e50fe2b None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:27.481197 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 20/37] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:29:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:29:27.491203 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39bdd40-ae36-4c0d-a2f0-9095197c04af None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:27.492407 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39bdd40-ae36-4c0d-a2f0-9095197c04af None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:27.493197 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39bdd40-ae36-4c0d-a2f0-9095197c04af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:27.496773 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa2183c-391f-46a9-ad1e-513f94006e2a None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:27.535905 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-f39bdd40-ae36-4c0d-a2f0-9095197c04af None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:29:27.538798 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-f39bdd40-ae36-4c0d-a2f0-9095197c04af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:29:27.572826 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feff668b-ab18-491f-ad86-d89934f9e0ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:27.574298 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feff668b-ab18-491f-ad86-d89934f9e0ae None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:27.574791 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feff668b-ab18-491f-ad86-d89934f9e0ae None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:27.575269 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feff668b-ab18-491f-ad86-d89934f9e0ae None admin] PATH_INFO: `/v3/users/12a220316b144048ad50f0affd87fa43` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:27.577169 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feff668b-ab18-491f-ad86-d89934f9e0ae None admin] RBAC: Authorizing `identity:get_user(user_id=12a220316b144048ad50f0affd87fa43)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:27.579264 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feff668b-ab18-491f-ad86-d89934f9e0ae None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:27.581581 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 18/38] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jun 16 17:29:27 2022] GET /identity/v3/users/12a220316b144048ad50f0affd87fa43 => generated 247 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:29:27.589762 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feff668b-ab18-491f-ad86-d89934f9e0ae None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:27.632734 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 21/39] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:29:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 17:29:27.658833 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39bdd40-ae36-4c0d-a2f0-9095197c04af None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:27.719067 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feff668b-ab18-491f-ad86-d89934f9e0ae None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:27.730507 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:27.748578 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:27.757206 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118e536f-f3f5-4eff-b594-b3d984b10d6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:27.758005 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118e536f-f3f5-4eff-b594-b3d984b10d6f None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:27.758292 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118e536f-f3f5-4eff-b594-b3d984b10d6f None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:27.758570 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118e536f-f3f5-4eff-b594-b3d984b10d6f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:27.759840 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118e536f-f3f5-4eff-b594-b3d984b10d6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:27.761336 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118e536f-f3f5-4eff-b594-b3d984b10d6f None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:27.763228 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 19/40] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jun 16 17:29:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:29:27.764498 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1279e0f-6a3e-41e0-886d-43e168614460 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:27.764790 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1279e0f-6a3e-41e0-886d-43e168614460 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:27.765068 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1279e0f-6a3e-41e0-886d-43e168614460 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:27.766355 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 20/41] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:29:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:29:27.769070 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1279e0f-6a3e-41e0-886d-43e168614460 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:27.783354 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39bdd40-ae36-4c0d-a2f0-9095197c04af None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:27.818498 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8450f78a-37df-47df-b285-12d516123868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:27.819193 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8450f78a-37df-47df-b285-12d516123868 None admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:27.819485 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8450f78a-37df-47df-b285-12d516123868 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:27.819765 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8450f78a-37df-47df-b285-12d516123868 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:27.820547 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8450f78a-37df-47df-b285-12d516123868 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:27.821945 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8450f78a-37df-47df-b285-12d516123868 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:27.835060 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 22/42] 10.222.0.42 () {60 vars in 1131 bytes} [Thu Jun 16 17:29:27 2022] POST /identity/v3/roles => generated 223 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:29:27.846319 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf699e6-c381-485c-bf07-d40f6b107c6a None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:27.846319 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf699e6-c381-485c-bf07-d40f6b107c6a None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:27.846319 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf699e6-c381-485c-bf07-d40f6b107c6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:27.855341 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-4bf699e6-c381-485c-bf07-d40f6b107c6a None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:29:27.856665 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-4bf699e6-c381-485c-bf07-d40f6b107c6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:29:27.899396 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1279e0f-6a3e-41e0-886d-43e168614460 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:27.941803 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5d57de-8211-4cd2-8a51-b4e07805a555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:27.942775 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5d57de-8211-4cd2-8a51-b4e07805a555 None admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:27.943065 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5d57de-8211-4cd2-8a51-b4e07805a555 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:27.943350 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5d57de-8211-4cd2-8a51-b4e07805a555 None admin] PATH_INFO: `/v3/domains/default/users/12a220316b144048ad50f0affd87fa43/roles/3ca2dcbbd29d4a31a3ab78f2299bc239` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:27.945498 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5d57de-8211-4cd2-8a51-b4e07805a555 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=12a220316b144048ad50f0affd87fa43, role_id=3ca2dcbbd29d4a31a3ab78f2299bc239)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:27.947279 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 23/43] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:29:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:29:27.947395 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5d57de-8211-4cd2-8a51-b4e07805a555 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:27.963139 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 21/44] 10.222.0.42 () {58 vars in 1361 bytes} [Thu Jun 16 17:29:27 2022] PUT /identity/v3/domains/default/users/12a220316b144048ad50f0affd87fa43/roles/3ca2dcbbd29d4a31a3ab78f2299bc239 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:29:27.971760 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf699e6-c381-485c-bf07-d40f6b107c6a None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:28.032894 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:28.103811 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf699e6-c381-485c-bf07-d40f6b107c6a None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:28.146038 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37777919-20d5-4345-a79c-e4f8fca271b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:28.146860 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37777919-20d5-4345-a79c-e4f8fca271b0 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:28.147250 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37777919-20d5-4345-a79c-e4f8fca271b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:28.147425 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37777919-20d5-4345-a79c-e4f8fca271b0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:28.148679 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37777919-20d5-4345-a79c-e4f8fca271b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:28.150143 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37777919-20d5-4345-a79c-e4f8fca271b0 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:28.151785 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 24/45] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jun 16 17:29:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:29:28.157858 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5d57de-8211-4cd2-8a51-b4e07805a555 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:28.216204 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:28.261473 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5d57de-8211-4cd2-8a51-b4e07805a555 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:28.295367 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ff6dff-d85d-4a67-95a2-68ba4bee1b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:28.296032 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ff6dff-d85d-4a67-95a2-68ba4bee1b4a None admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:28.296322 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ff6dff-d85d-4a67-95a2-68ba4bee1b4a None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:28.296627 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ff6dff-d85d-4a67-95a2-68ba4bee1b4a None admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:28.297976 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ff6dff-d85d-4a67-95a2-68ba4bee1b4a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:28.299524 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ff6dff-d85d-4a67-95a2-68ba4bee1b4a None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:28.325499 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 22/46] 10.222.0.42 () {60 vars in 1141 bytes} [Thu Jun 16 17:29:28 2022] POST /identity/v3/projects => generated 312 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:29:30.303353 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d021ef-0d07-4f2d-802a-f67003ea06fa None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:30.303870 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d021ef-0d07-4f2d-802a-f67003ea06fa None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:30.304073 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d021ef-0d07-4f2d-802a-f67003ea06fa None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:30.305462 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 25/47] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:29:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:29:30.310854 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bf411c-d710-42e9-aa0e-d6209f6ede25 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:30.311246 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bf411c-d710-42e9-aa0e-d6209f6ede25 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:30.311592 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bf411c-d710-42e9-aa0e-d6209f6ede25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:30.330796 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-d4bf411c-d710-42e9-aa0e-d6209f6ede25 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:29:30.332285 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-d4bf411c-d710-42e9-aa0e-d6209f6ede25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:29:30.400459 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 23/48] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:29:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:29:30.425140 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d021ef-0d07-4f2d-802a-f67003ea06fa None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:30.508333 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7fdc4e-4b05-4390-99c5-a74de173bd4a None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:30.508672 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7fdc4e-4b05-4390-99c5-a74de173bd4a None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:30.508957 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7fdc4e-4b05-4390-99c5-a74de173bd4a None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:30.510348 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 24/49] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:29:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:29:30.515508 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a0366d-058d-453c-ad4f-f028cd6fb438 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:30.515815 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a0366d-058d-453c-ad4f-f028cd6fb438 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:30.516125 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a0366d-058d-453c-ad4f-f028cd6fb438 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:30.533543 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-d2a0366d-058d-453c-ad4f-f028cd6fb438 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:29:30.534799 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-d2a0366d-058d-453c-ad4f-f028cd6fb438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:29:30.544008 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d021ef-0d07-4f2d-802a-f67003ea06fa None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:30.558846 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:30.566183 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:30.581969 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae7537e-ff62-4ca3-a8f2-427ff981b7de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:30.582892 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae7537e-ff62-4ca3-a8f2-427ff981b7de None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:30.583138 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae7537e-ff62-4ca3-a8f2-427ff981b7de None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:30.583389 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae7537e-ff62-4ca3-a8f2-427ff981b7de None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:30.594247 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae7537e-ff62-4ca3-a8f2-427ff981b7de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:29:30.602556 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-9ae7537e-ff62-4ca3-a8f2-427ff981b7de None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:29:30.602556 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:29:30.602556 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:29:30.602556 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:29:30.602556 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:29:30.602556 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:29:30.602556 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:29:30.602556 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:29:30.602556 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:29:30.602556 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:29:30.602556 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:29:30.602556 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:29:30.602556 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:29:30.602556 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:29:30.602556 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:29:30.602556 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:29:30.603377 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:29:30.603377 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:29:30.603377 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:29:30.603377 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:29:30.603377 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:29:30.603377 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:29:30.603377 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:29:30.603377 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:29:30.603377 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:29:30.603377 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:29:30.603377 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:29:30.603377 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:29:30.603377 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:29:30.603377 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:29:30.603377 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:29:30.603377 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:29:30.604119 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:29:30.604119 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:29:30.604119 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:29:30.604119 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:29:30.604119 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:29:30.604119 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:29:30.604119 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:29:30.604119 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:29:30.604119 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:29:30.604119 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:29:30.604119 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:29:30.604119 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:29:30.604119 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 26/50] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Jun 16 17:29:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:29:30.606023 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 25/51] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:29:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 17:29:30.609780 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae7537e-ff62-4ca3-a8f2-427ff981b7de None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:30.630026 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a0366d-058d-453c-ad4f-f028cd6fb438 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:30.713749 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae7537e-ff62-4ca3-a8f2-427ff981b7de None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:30.731758 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a0366d-058d-453c-ad4f-f028cd6fb438 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:30.751213 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3ba07d-e531-4255-a69b-011067bbe9a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:30.752045 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3ba07d-e531-4255-a69b-011067bbe9a1 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:30.752361 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3ba07d-e531-4255-a69b-011067bbe9a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:30.752725 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3ba07d-e531-4255-a69b-011067bbe9a1 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:30.753575 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3ba07d-e531-4255-a69b-011067bbe9a1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:29:30.753929 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3ba07d-e531-4255-a69b-011067bbe9a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:30.756033 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3ba07d-e531-4255-a69b-011067bbe9a1 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:30.765269 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 27/52] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jun 16 17:29:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:29:30.772410 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3ba07d-e531-4255-a69b-011067bbe9a1 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:30.772656 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b78815-ef30-4295-b460-4e2cd592ae77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:30.773457 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b78815-ef30-4295-b460-4e2cd592ae77 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:30.773737 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b78815-ef30-4295-b460-4e2cd592ae77 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:30.774146 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b78815-ef30-4295-b460-4e2cd592ae77 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:30.775419 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b78815-ef30-4295-b460-4e2cd592ae77 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:30.776931 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b78815-ef30-4295-b460-4e2cd592ae77 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:30.778565 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 26/53] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jun 16 17:29:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:29:30.784657 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b78815-ef30-4295-b460-4e2cd592ae77 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:30.814341 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:30.822319 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:30.887774 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3ba07d-e531-4255-a69b-011067bbe9a1 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:30.896449 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b78815-ef30-4295-b460-4e2cd592ae77 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:30.926822 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea96f8ca-9eb5-4a05-9c29-cb86e942e183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:30.927689 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea96f8ca-9eb5-4a05-9c29-cb86e942e183 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:30.927977 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea96f8ca-9eb5-4a05-9c29-cb86e942e183 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:30.928255 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea96f8ca-9eb5-4a05-9c29-cb86e942e183 None admin] PATH_INFO: `/v3/users/12a220316b144048ad50f0affd87fa43` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:30.930043 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea96f8ca-9eb5-4a05-9c29-cb86e942e183 None admin] RBAC: Authorizing `identity:get_user(user_id=12a220316b144048ad50f0affd87fa43)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:30.931506 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea96f8ca-9eb5-4a05-9c29-cb86e942e183 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:30.933186 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 28/54] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jun 16 17:29:30 2022] GET /identity/v3/users/12a220316b144048ad50f0affd87fa43 => generated 247 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:29:30.939179 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea96f8ca-9eb5-4a05-9c29-cb86e942e183 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:30.940949 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06f383d-43a8-467b-94d7-eee5542ab82e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:30.941626 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06f383d-43a8-467b-94d7-eee5542ab82e None admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:30.941917 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06f383d-43a8-467b-94d7-eee5542ab82e None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:30.942197 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06f383d-43a8-467b-94d7-eee5542ab82e None admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:30.942909 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06f383d-43a8-467b-94d7-eee5542ab82e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:30.944401 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06f383d-43a8-467b-94d7-eee5542ab82e None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:30.972838 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 27/55] 10.222.0.42 () {60 vars in 1140 bytes} [Thu Jun 16 17:29:30 2022] POST /identity/v3/projects => generated 298 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:29:31.045688 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea96f8ca-9eb5-4a05-9c29-cb86e942e183 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:31.049433 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:31.096799 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b16fafa-617f-4d1d-af9b-5ef3d47f869a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:31.097047 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b16fafa-617f-4d1d-af9b-5ef3d47f869a None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:31.097340 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b16fafa-617f-4d1d-af9b-5ef3d47f869a None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:31.097685 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b16fafa-617f-4d1d-af9b-5ef3d47f869a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:31.099116 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b16fafa-617f-4d1d-af9b-5ef3d47f869a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:31.100519 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b16fafa-617f-4d1d-af9b-5ef3d47f869a None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:31.102307 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 29/56] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jun 16 17:29:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:29:31.108352 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06f383d-43a8-467b-94d7-eee5542ab82e None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:31.206151 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:31.221366 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06f383d-43a8-467b-94d7-eee5542ab82e None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:31.255381 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5839cc1-a24a-436f-ba46-8e06cbf9b96c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:31.256047 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5839cc1-a24a-436f-ba46-8e06cbf9b96c None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:31.256341 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5839cc1-a24a-436f-ba46-8e06cbf9b96c None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:31.256651 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5839cc1-a24a-436f-ba46-8e06cbf9b96c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:31.257501 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5839cc1-a24a-436f-ba46-8e06cbf9b96c None admin] RBAC: Adding query filter params (role.id=3ca2dcbbd29d4a31a3ab78f2299bc239, scope.domain.id=default, user.id=12a220316b144048ad50f0affd87fa43) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:29:31.257816 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5839cc1-a24a-436f-ba46-8e06cbf9b96c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:31.259287 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5839cc1-a24a-436f-ba46-8e06cbf9b96c None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:31.265492 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 28/57] 10.222.0.42 () {56 vars in 1322 bytes} [Thu Jun 16 17:29:31 2022] GET /identity/v3/role_assignments?user.id=12a220316b144048ad50f0affd87fa43&scope.domain.id=default&role.id=3ca2dcbbd29d4a31a3ab78f2299bc239 => generated 530 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:29:33.170791 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fcb328-6f0e-44e2-9b84-a73177398cb2 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:33.171295 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fcb328-6f0e-44e2-9b84-a73177398cb2 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:33.171396 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fcb328-6f0e-44e2-9b84-a73177398cb2 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:33.172811 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 30/58] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:29:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:29:33.177920 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b461cf-da77-45a4-88fb-0691ba45826f None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:33.178388 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b461cf-da77-45a4-88fb-0691ba45826f None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:33.178491 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b461cf-da77-45a4-88fb-0691ba45826f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:33.195421 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-73b461cf-da77-45a4-88fb-0691ba45826f None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:29:33.196714 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-73b461cf-da77-45a4-88fb-0691ba45826f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:29:33.271243 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 29/59] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:29:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:29:33.295983 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fcb328-6f0e-44e2-9b84-a73177398cb2 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:33.402467 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fcb328-6f0e-44e2-9b84-a73177398cb2 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:33.441767 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4790500a-702c-46e8-8c54-072cc7165915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:33.442534 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4790500a-702c-46e8-8c54-072cc7165915 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:33.442821 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4790500a-702c-46e8-8c54-072cc7165915 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:33.443138 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4790500a-702c-46e8-8c54-072cc7165915 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:33.444412 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4790500a-702c-46e8-8c54-072cc7165915 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:33.445946 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4790500a-702c-46e8-8c54-072cc7165915 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:33.447592 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 31/60] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jun 16 17:29:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:29:33.453243 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b461cf-da77-45a4-88fb-0691ba45826f None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:33.533497 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:33.551563 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b461cf-da77-45a4-88fb-0691ba45826f None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:33.585501 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7241437-bc1e-4a70-b2bf-0ef72165d007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:33.586166 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7241437-bc1e-4a70-b2bf-0ef72165d007 None admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:33.586459 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7241437-bc1e-4a70-b2bf-0ef72165d007 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:33.586743 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7241437-bc1e-4a70-b2bf-0ef72165d007 None admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:33.587487 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7241437-bc1e-4a70-b2bf-0ef72165d007 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:33.589000 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7241437-bc1e-4a70-b2bf-0ef72165d007 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:33.614294 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 30/61] 10.222.0.42 () {60 vars in 1132 bytes} [Thu Jun 16 17:29:33 2022] POST /identity/v3/users => generated 275 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:29:36.826779 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd999fe-cb38-422c-94d8-3e4c9b0be036 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:36.827287 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd999fe-cb38-422c-94d8-3e4c9b0be036 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:36.827359 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd999fe-cb38-422c-94d8-3e4c9b0be036 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:36.828790 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 32/62] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:29:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:29:36.834302 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adff2434-73ab-472e-85c6-138b2dbd0311 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:36.835169 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adff2434-73ab-472e-85c6-138b2dbd0311 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:36.835653 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adff2434-73ab-472e-85c6-138b2dbd0311 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:36.861908 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-adff2434-73ab-472e-85c6-138b2dbd0311 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:29:36.864232 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-adff2434-73ab-472e-85c6-138b2dbd0311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:29:36.970535 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 31/63] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:29:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:29:37.018824 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd999fe-cb38-422c-94d8-3e4c9b0be036 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:37.090972 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:37.140016 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c82f772-08ba-4894-a6ea-29c78f7cf3df None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:37.140706 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c82f772-08ba-4894-a6ea-29c78f7cf3df None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:37.141715 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c82f772-08ba-4894-a6ea-29c78f7cf3df None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:37.143561 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 32/64] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:29:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:29:37.149177 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4bd29d-388e-486f-998b-568e17c142d6 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:37.149483 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4bd29d-388e-486f-998b-568e17c142d6 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:37.149758 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4bd29d-388e-486f-998b-568e17c142d6 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:37.151387 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 33/65] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:29:37 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:29:37.152771 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c265395b-5140-4aa2-897c-278aed6e3a38 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:37.153263 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c265395b-5140-4aa2-897c-278aed6e3a38 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:37.153789 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c265395b-5140-4aa2-897c-278aed6e3a38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:37.173745 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-c265395b-5140-4aa2-897c-278aed6e3a38 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:29:37.175102 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd999fe-cb38-422c-94d8-3e4c9b0be036 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:37.175595 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-c265395b-5140-4aa2-897c-278aed6e3a38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:29:37.212162 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c310ac6-e051-4845-aa52-b835ee580d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:37.213017 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c310ac6-e051-4845-aa52-b835ee580d96 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:37.213298 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c310ac6-e051-4845-aa52-b835ee580d96 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:37.213745 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c310ac6-e051-4845-aa52-b835ee580d96 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:37.225752 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c310ac6-e051-4845-aa52-b835ee580d96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:29:37.233823 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-9c310ac6-e051-4845-aa52-b835ee580d96 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:29:37.233823 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:29:37.233823 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:29:37.233823 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:29:37.233823 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:29:37.233823 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:29:37.233823 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:29:37.233823 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:29:37.233823 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:29:37.233823 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:29:37.233823 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:29:37.233823 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:29:37.233823 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:29:37.233823 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:29:37.233823 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:29:37.233823 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:29:37.234692 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:29:37.234692 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:29:37.234692 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:29:37.234692 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:29:37.234692 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:29:37.234692 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:29:37.234692 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:29:37.234692 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:29:37.234692 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:29:37.234692 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:29:37.234692 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:29:37.234692 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:29:37.234692 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:29:37.234692 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:29:37.234692 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:29:37.234692 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:29:37.236169 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:29:37.236169 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:29:37.236169 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:29:37.236169 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:29:37.236169 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:29:37.236169 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:29:37.236169 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:29:37.236169 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:29:37.236169 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:29:37.236169 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:29:37.236169 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:29:37.236169 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:29:37.238592 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 33/66] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jun 16 17:29:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:29:37.239995 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19caee1f-19ad-4b16-96e2-eaa19f240b93 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:37.240496 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19caee1f-19ad-4b16-96e2-eaa19f240b93 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:37.240955 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19caee1f-19ad-4b16-96e2-eaa19f240b93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:37.247510 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:37.256557 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 34/67] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:29:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:29:37.257560 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c265395b-5140-4aa2-897c-278aed6e3a38 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:37.259798 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-19caee1f-19ad-4b16-96e2-eaa19f240b93 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:29:37.261268 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-19caee1f-19ad-4b16-96e2-eaa19f240b93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:29:37.342031 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 34/68] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:29:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:29:37.343082 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19caee1f-19ad-4b16-96e2-eaa19f240b93 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:37.381108 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:37.389831 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c265395b-5140-4aa2-897c-278aed6e3a38 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:37.436681 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726ea3fc-a704-4fcd-bd8d-eb070d9afc43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:37.437361 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726ea3fc-a704-4fcd-bd8d-eb070d9afc43 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:37.437648 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726ea3fc-a704-4fcd-bd8d-eb070d9afc43 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:37.438483 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726ea3fc-a704-4fcd-bd8d-eb070d9afc43 None admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:37.439274 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726ea3fc-a704-4fcd-bd8d-eb070d9afc43 None admin] RBAC: Adding query filter params (name=member) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:29:37.439599 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726ea3fc-a704-4fcd-bd8d-eb070d9afc43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:37.441503 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726ea3fc-a704-4fcd-bd8d-eb070d9afc43 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:37.453281 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19caee1f-19ad-4b16-96e2-eaa19f240b93 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:37.457658 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 35/69] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Jun 16 17:29:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:29:37.458523 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726ea3fc-a704-4fcd-bd8d-eb070d9afc43 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:37.491259 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1b19dc-deae-4bb1-9fa6-1642502c45f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:37.492901 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1b19dc-deae-4bb1-9fa6-1642502c45f7 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:37.493338 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1b19dc-deae-4bb1-9fa6-1642502c45f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:37.494013 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1b19dc-deae-4bb1-9fa6-1642502c45f7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:37.508832 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1b19dc-deae-4bb1-9fa6-1642502c45f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:29:37.520318 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-2d1b19dc-deae-4bb1-9fa6-1642502c45f7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:29:37.520318 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:29:37.520318 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:29:37.520318 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:29:37.520318 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:29:37.520318 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:29:37.520318 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:29:37.520318 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:29:37.520318 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:29:37.520318 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:29:37.520318 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:29:37.520318 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:29:37.520318 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:29:37.520318 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:29:37.520318 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:29:37.520318 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:29:37.521432 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:29:37.521432 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:29:37.521432 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:29:37.521432 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:29:37.521432 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:29:37.521432 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:29:37.521432 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:29:37.521432 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:29:37.521432 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:29:37.521432 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:29:37.521432 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:29:37.521432 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:29:37.521432 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:29:37.521432 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:29:37.521432 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:29:37.521432 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:29:37.522393 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:29:37.522393 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:29:37.522393 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:29:37.522393 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:29:37.522393 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:29:37.522393 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:29:37.522393 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:29:37.522393 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:29:37.522393 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:29:37.522393 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:29:37.522393 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:29:37.522393 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:29:37.525290 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 35/70] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Jun 16 17:29:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:29:37.526406 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1b19dc-deae-4bb1-9fa6-1642502c45f7 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:37.540740 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:37.596416 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726ea3fc-a704-4fcd-bd8d-eb070d9afc43 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:37.634188 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:37.639231 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a304f0-ecfe-4b50-a6bd-8a3ceb2d5cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:37.640082 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a304f0-ecfe-4b50-a6bd-8a3ceb2d5cf3 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:37.640390 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a304f0-ecfe-4b50-a6bd-8a3ceb2d5cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:37.640677 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a304f0-ecfe-4b50-a6bd-8a3ceb2d5cf3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:37.646149 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1b19dc-deae-4bb1-9fa6-1642502c45f7 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:37.654151 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a304f0-ecfe-4b50-a6bd-8a3ceb2d5cf3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:29:37.663444 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-96a304f0-ecfe-4b50-a6bd-8a3ceb2d5cf3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 17:29:37.663444 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:29:37.663444 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:29:37.663444 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:29:37.663444 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:29:37.663444 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:29:37.663444 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:29:37.663444 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:29:37.663444 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:29:37.663444 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:29:37.663444 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:29:37.663444 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:29:37.663444 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:29:37.663444 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:29:37.663444 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:29:37.663444 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:29:37.664409 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:29:37.664409 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:29:37.664409 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:29:37.664409 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:29:37.664409 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:29:37.664409 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:29:37.664409 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:29:37.664409 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:29:37.664409 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:29:37.664409 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:29:37.664409 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:29:37.664409 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:29:37.664409 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:29:37.664409 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:29:37.664409 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:29:37.664409 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:29:37.665187 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:29:37.665187 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:29:37.665187 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:29:37.665187 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:29:37.665187 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:29:37.665187 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:29:37.665187 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:29:37.665187 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:29:37.665187 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:29:37.665187 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:29:37.665187 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 17:29:37.665187 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:29:37.665187 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 36/71] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Jun 16 17:29:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:29:37.665810 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a304f0-ecfe-4b50-a6bd-8a3ceb2d5cf3 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:37.688795 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae7c791-73cc-4500-ab5c-b38a7f441bb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:37.689431 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae7c791-73cc-4500-ab5c-b38a7f441bb4 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:37.689714 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae7c791-73cc-4500-ab5c-b38a7f441bb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:37.689992 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae7c791-73cc-4500-ab5c-b38a7f441bb4 None admin] PATH_INFO: `/v3/users/f72678162d55418190ac5579cb1eb41f` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:37.707448 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae7c791-73cc-4500-ab5c-b38a7f441bb4 None admin] RBAC: Authorizing `identity:get_user(user_id=f72678162d55418190ac5579cb1eb41f)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:37.708972 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae7c791-73cc-4500-ab5c-b38a7f441bb4 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:37.715096 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 36/72] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jun 16 17:29:37 2022] GET /identity/v3/users/f72678162d55418190ac5579cb1eb41f => generated 275 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:29:37.715096 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bae465a-8652-4b09-a811-3907b9b74f57 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:37.715096 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bae465a-8652-4b09-a811-3907b9b74f57 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:37.715096 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bae465a-8652-4b09-a811-3907b9b74f57 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:37.715096 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 37/73] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:29:37 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:29:37.715096 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bae465a-8652-4b09-a811-3907b9b74f57 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:37.779319 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a304f0-ecfe-4b50-a6bd-8a3ceb2d5cf3 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:37.799356 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:37.814066 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3034fcb8-1ea8-4b37-a3be-6464b179b87b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:37.814743 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3034fcb8-1ea8-4b37-a3be-6464b179b87b None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:37.815032 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3034fcb8-1ea8-4b37-a3be-6464b179b87b None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:37.815314 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3034fcb8-1ea8-4b37-a3be-6464b179b87b None admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:37.816081 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3034fcb8-1ea8-4b37-a3be-6464b179b87b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:29:37.816418 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3034fcb8-1ea8-4b37-a3be-6464b179b87b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:37.817814 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3034fcb8-1ea8-4b37-a3be-6464b179b87b None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:37.820242 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bae465a-8652-4b09-a811-3907b9b74f57 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:37.826141 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 37/74] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jun 16 17:29:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:29:37.826968 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3034fcb8-1ea8-4b37-a3be-6464b179b87b None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:37.863724 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96de03c-e9fe-4508-987b-6dda701ee9c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:37.864444 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96de03c-e9fe-4508-987b-6dda701ee9c3 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:37.864741 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96de03c-e9fe-4508-987b-6dda701ee9c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:37.865022 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96de03c-e9fe-4508-987b-6dda701ee9c3 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:37.865781 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96de03c-e9fe-4508-987b-6dda701ee9c3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:29:37.866093 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96de03c-e9fe-4508-987b-6dda701ee9c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:37.867503 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96de03c-e9fe-4508-987b-6dda701ee9c3 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:37.876869 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 38/75] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jun 16 17:29:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:29:37.878338 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827d0c47-f892-4e41-a8e1-3333f2fabb79 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:37.878633 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827d0c47-f892-4e41-a8e1-3333f2fabb79 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:37.878918 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827d0c47-f892-4e41-a8e1-3333f2fabb79 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:37.880643 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 39/76] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:29:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:29:37.886366 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0946e864-8e07-4aae-96a8-f3b5a6a06dc2 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:37.886632 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0946e864-8e07-4aae-96a8-f3b5a6a06dc2 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:37.886921 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0946e864-8e07-4aae-96a8-f3b5a6a06dc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:37.905506 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-0946e864-8e07-4aae-96a8-f3b5a6a06dc2 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:29:37.907296 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-0946e864-8e07-4aae-96a8-f3b5a6a06dc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:29:37.910727 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:37.932292 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3034fcb8-1ea8-4b37-a3be-6464b179b87b None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:37.968436 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7bd017-165c-418d-acc2-92cea6a78e6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:37.970583 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7bd017-165c-418d-acc2-92cea6a78e6e None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:37.970583 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7bd017-165c-418d-acc2-92cea6a78e6e None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:37.970583 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7bd017-165c-418d-acc2-92cea6a78e6e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:37.983486 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 40/77] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:29:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:29:37.984586 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0946e864-8e07-4aae-96a8-f3b5a6a06dc2 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:38.030526 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:38.050328 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7bd017-165c-418d-acc2-92cea6a78e6e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:38.051844 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7bd017-165c-418d-acc2-92cea6a78e6e None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:38.086360 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0946e864-8e07-4aae-96a8-f3b5a6a06dc2 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:38.121714 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab225c0-19a0-4eed-9f5f-a991f45101ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:38.122514 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab225c0-19a0-4eed-9f5f-a991f45101ed None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:38.122799 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab225c0-19a0-4eed-9f5f-a991f45101ed None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:38.123080 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab225c0-19a0-4eed-9f5f-a991f45101ed None admin] PATH_INFO: `/v3/users/12a220316b144048ad50f0affd87fa43` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:38.124281 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab225c0-19a0-4eed-9f5f-a991f45101ed None admin] RBAC: Authorizing `identity:get_user(user_id=12a220316b144048ad50f0affd87fa43)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:38.125829 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab225c0-19a0-4eed-9f5f-a991f45101ed None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:38.127471 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 41/78] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jun 16 17:29:37 2022] GET /identity/v3/users/12a220316b144048ad50f0affd87fa43 => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:29:38.132052 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab225c0-19a0-4eed-9f5f-a991f45101ed None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:38.140413 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:38.151692 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 38/79] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jun 16 17:29:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:29:38.153103 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91feea05-cd00-405e-880c-6d56264d9ebf None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:38.153401 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91feea05-cd00-405e-880c-6d56264d9ebf None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:38.153776 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91feea05-cd00-405e-880c-6d56264d9ebf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:38.172050 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-91feea05-cd00-405e-880c-6d56264d9ebf None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:29:38.174486 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-91feea05-cd00-405e-880c-6d56264d9ebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:29:38.233512 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab225c0-19a0-4eed-9f5f-a991f45101ed None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:38.249191 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 39/80] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:29:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:29:38.250048 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91feea05-cd00-405e-880c-6d56264d9ebf None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:38.271135 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25f854d-413d-4382-b76b-d4a574481da9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:38.272594 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25f854d-413d-4382-b76b-d4a574481da9 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:38.272942 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25f854d-413d-4382-b76b-d4a574481da9 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:38.273156 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25f854d-413d-4382-b76b-d4a574481da9 None admin] PATH_INFO: `/v3/users/f72678162d55418190ac5579cb1eb41f` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:38.274795 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25f854d-413d-4382-b76b-d4a574481da9 None admin] RBAC: Authorizing `identity:get_user(user_id=f72678162d55418190ac5579cb1eb41f)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:38.276607 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25f854d-413d-4382-b76b-d4a574481da9 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:38.277781 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:38.278310 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 42/81] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jun 16 17:29:38 2022] GET /identity/v3/users/f72678162d55418190ac5579cb1eb41f => generated 275 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:29:38.279120 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25f854d-413d-4382-b76b-d4a574481da9 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:38.357721 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:38.367654 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91feea05-cd00-405e-880c-6d56264d9ebf None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:38.383031 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25f854d-413d-4382-b76b-d4a574481da9 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:38.403951 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36ee316-4321-4fb3-9e87-014c796bd5f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:38.404800 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36ee316-4321-4fb3-9e87-014c796bd5f5 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:38.405088 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36ee316-4321-4fb3-9e87-014c796bd5f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:38.405370 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36ee316-4321-4fb3-9e87-014c796bd5f5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:38.415710 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36ee316-4321-4fb3-9e87-014c796bd5f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:29:38.417193 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2995e7f-7c98-42ae-a605-da38b4fc8554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:38.417846 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2995e7f-7c98-42ae-a605-da38b4fc8554 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:38.418131 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2995e7f-7c98-42ae-a605-da38b4fc8554 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:38.418410 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2995e7f-7c98-42ae-a605-da38b4fc8554 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:38.424062 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-b36ee316-4321-4fb3-9e87-014c796bd5f5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:29:38.424062 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:29:38.424062 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:29:38.424062 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:29:38.424062 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:29:38.424062 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:29:38.424062 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:29:38.424062 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:29:38.424062 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:29:38.424062 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:29:38.424062 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:29:38.424062 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:29:38.424062 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:29:38.424062 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:29:38.424062 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:29:38.424062 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:29:38.425137 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:29:38.425137 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:29:38.425137 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:29:38.425137 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:29:38.425137 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:29:38.425137 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:29:38.425137 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:29:38.425137 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:29:38.425137 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:29:38.425137 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:29:38.425137 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:29:38.425137 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:29:38.425137 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:29:38.425137 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:29:38.425137 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:29:38.425137 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:29:38.425908 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:29:38.425908 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:29:38.425908 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:29:38.425908 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:29:38.425908 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:29:38.425908 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:29:38.425908 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:29:38.425908 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:29:38.425908 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:29:38.425908 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:29:38.425908 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:29:38.425908 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:29:38.425908 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 40/82] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jun 16 17:29:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:29:38.426532 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36ee316-4321-4fb3-9e87-014c796bd5f5 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:38.497190 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2995e7f-7c98-42ae-a605-da38b4fc8554 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:38.498670 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2995e7f-7c98-42ae-a605-da38b4fc8554 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:38.500355 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:38.530464 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36ee316-4321-4fb3-9e87-014c796bd5f5 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:38.565191 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7b5876-92fb-4e4f-8f2d-45bb931213a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:38.566077 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7b5876-92fb-4e4f-8f2d-45bb931213a2 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:38.566363 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7b5876-92fb-4e4f-8f2d-45bb931213a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:38.566643 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7b5876-92fb-4e4f-8f2d-45bb931213a2 None admin] PATH_INFO: `/v3/projects/3be66087f3954ee4bac1dc53ea4d59ed` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:38.567847 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7b5876-92fb-4e4f-8f2d-45bb931213a2 None admin] RBAC: Authorizing `identity:get_project(project_id=3be66087f3954ee4bac1dc53ea4d59ed)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:38.569355 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7b5876-92fb-4e4f-8f2d-45bb931213a2 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:38.571071 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 41/83] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jun 16 17:29:38 2022] GET /identity/v3/projects/3be66087f3954ee4bac1dc53ea4d59ed => generated 298 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:29:38.571848 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7b5876-92fb-4e4f-8f2d-45bb931213a2 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:38.578968 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:38.603601 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 43/84] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jun 16 17:29:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:29:38.604479 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2995e7f-7c98-42ae-a605-da38b4fc8554 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:38.675360 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7b5876-92fb-4e4f-8f2d-45bb931213a2 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:38.706704 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2995e7f-7c98-42ae-a605-da38b4fc8554 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:38.710320 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50db018-07d7-41ea-924e-6d2f90bd7415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:38.711120 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50db018-07d7-41ea-924e-6d2f90bd7415 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:38.711405 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50db018-07d7-41ea-924e-6d2f90bd7415 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:38.711684 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50db018-07d7-41ea-924e-6d2f90bd7415 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:38.712875 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50db018-07d7-41ea-924e-6d2f90bd7415 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:38.714337 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50db018-07d7-41ea-924e-6d2f90bd7415 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:38.715970 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 42/85] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jun 16 17:29:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:29:38.716765 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50db018-07d7-41ea-924e-6d2f90bd7415 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:38.728323 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:38.743383 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db09b66-5577-4c92-876b-b9c232e75764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:38.744045 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db09b66-5577-4c92-876b-b9c232e75764 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:38.744329 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db09b66-5577-4c92-876b-b9c232e75764 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:38.744644 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db09b66-5577-4c92-876b-b9c232e75764 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:38.791695 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:38.815901 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db09b66-5577-4c92-876b-b9c232e75764 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:38.817427 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db09b66-5577-4c92-876b-b9c232e75764 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:38.822113 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50db018-07d7-41ea-924e-6d2f90bd7415 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:38.855685 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1c79c2-8de2-4d3c-ad85-56db42e4c453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:38.856362 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1c79c2-8de2-4d3c-ad85-56db42e4c453 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:38.856709 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1c79c2-8de2-4d3c-ad85-56db42e4c453 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:38.856994 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1c79c2-8de2-4d3c-ad85-56db42e4c453 None admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:38.857750 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1c79c2-8de2-4d3c-ad85-56db42e4c453 None admin] RBAC: Adding query filter params (name=member) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:29:38.858061 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1c79c2-8de2-4d3c-ad85-56db42e4c453 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:38.859461 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1c79c2-8de2-4d3c-ad85-56db42e4c453 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:38.868760 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 43/86] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Jun 16 17:29:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:29:38.869612 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1c79c2-8de2-4d3c-ad85-56db42e4c453 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:38.926930 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 44/87] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jun 16 17:29:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:29:38.927719 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db09b66-5577-4c92-876b-b9c232e75764 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:38.951706 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:38.986160 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1c79c2-8de2-4d3c-ad85-56db42e4c453 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:39.006977 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:39.019455 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bc7a5f-969c-4c6e-b2f5-9815a731bb2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:39.020118 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bc7a5f-969c-4c6e-b2f5-9815a731bb2a None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:39.020433 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bc7a5f-969c-4c6e-b2f5-9815a731bb2a None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:39.020719 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bc7a5f-969c-4c6e-b2f5-9815a731bb2a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:39.021563 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bc7a5f-969c-4c6e-b2f5-9815a731bb2a None admin] RBAC: Adding query filter params (role.id=0be2075946f54c8485f1c275f746478a, scope.project.id=3be66087f3954ee4bac1dc53ea4d59ed, user.id=f72678162d55418190ac5579cb1eb41f) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:29:39.022364 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bc7a5f-969c-4c6e-b2f5-9815a731bb2a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:39.023839 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bc7a5f-969c-4c6e-b2f5-9815a731bb2a None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:39.029382 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 44/88] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jun 16 17:29:38 2022] GET /identity/v3/role_assignments?user.id=f72678162d55418190ac5579cb1eb41f&scope.project.id=3be66087f3954ee4bac1dc53ea4d59ed&role.id=0be2075946f54c8485f1c275f746478a => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:29:39.030158 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bc7a5f-969c-4c6e-b2f5-9815a731bb2a None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:39.030470 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db09b66-5577-4c92-876b-b9c232e75764 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:39.082507 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c22ec5-8381-4947-bf2d-aa0dbf854f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:39.083396 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c22ec5-8381-4947-bf2d-aa0dbf854f36 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:39.083693 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c22ec5-8381-4947-bf2d-aa0dbf854f36 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:39.083964 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c22ec5-8381-4947-bf2d-aa0dbf854f36 None admin] PATH_INFO: `/v3/projects/3be66087f3954ee4bac1dc53ea4d59ed` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:39.085299 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c22ec5-8381-4947-bf2d-aa0dbf854f36 None admin] RBAC: Authorizing `identity:get_project(project_id=3be66087f3954ee4bac1dc53ea4d59ed)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:39.086827 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c22ec5-8381-4947-bf2d-aa0dbf854f36 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:39.088705 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 45/89] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jun 16 17:29:38 2022] GET /identity/v3/projects/3be66087f3954ee4bac1dc53ea4d59ed => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:29:39.089537 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c22ec5-8381-4947-bf2d-aa0dbf854f36 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:39.171553 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bc7a5f-969c-4c6e-b2f5-9815a731bb2a None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:39.210496 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c22ec5-8381-4947-bf2d-aa0dbf854f36 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:39.215243 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66db4db-a243-4dbb-af14-59f2e80de3fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:39.216009 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66db4db-a243-4dbb-af14-59f2e80de3fa None admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:39.216221 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66db4db-a243-4dbb-af14-59f2e80de3fa None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:39.216599 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66db4db-a243-4dbb-af14-59f2e80de3fa None admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:39.217250 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66db4db-a243-4dbb-af14-59f2e80de3fa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:39.218719 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66db4db-a243-4dbb-af14-59f2e80de3fa None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:39.226795 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:39.243937 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 45/90] 10.222.0.42 () {60 vars in 1132 bytes} [Thu Jun 16 17:29:39 2022] POST /identity/v3/users => generated 289 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:29:39.244748 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66db4db-a243-4dbb-af14-59f2e80de3fa None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:39.265763 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ccc690-efe6-4209-9121-266c3446225a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:39.267893 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ccc690-efe6-4209-9121-266c3446225a None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:39.268181 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ccc690-efe6-4209-9121-266c3446225a None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:39.268536 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ccc690-efe6-4209-9121-266c3446225a None admin] PATH_INFO: `/v3/users/f72678162d55418190ac5579cb1eb41f` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:39.269988 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ccc690-efe6-4209-9121-266c3446225a None admin] RBAC: Authorizing `identity:get_user(user_id=f72678162d55418190ac5579cb1eb41f)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:39.271475 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ccc690-efe6-4209-9121-266c3446225a None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:39.277635 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 46/91] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jun 16 17:29:39 2022] GET /identity/v3/users/f72678162d55418190ac5579cb1eb41f => generated 275 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:29:39.277635 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ccc690-efe6-4209-9121-266c3446225a None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:39.299223 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:39.387315 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66db4db-a243-4dbb-af14-59f2e80de3fa None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:39.429199 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7519e00b-a5b5-4aeb-8bab-2ecff95f7a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:39.430366 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7519e00b-a5b5-4aeb-8bab-2ecff95f7a3d None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:39.430798 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7519e00b-a5b5-4aeb-8bab-2ecff95f7a3d None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:39.431237 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7519e00b-a5b5-4aeb-8bab-2ecff95f7a3d None admin] PATH_INFO: `/v3/projects/3be66087f3954ee4bac1dc53ea4d59ed` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:39.432814 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7519e00b-a5b5-4aeb-8bab-2ecff95f7a3d None admin] RBAC: Authorizing `identity:get_project(project_id=3be66087f3954ee4bac1dc53ea4d59ed)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:39.434463 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7519e00b-a5b5-4aeb-8bab-2ecff95f7a3d None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:39.436389 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 46/92] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jun 16 17:29:39 2022] GET /identity/v3/projects/3be66087f3954ee4bac1dc53ea4d59ed => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:29:39.438908 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7519e00b-a5b5-4aeb-8bab-2ecff95f7a3d None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:39.446417 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ccc690-efe6-4209-9121-266c3446225a None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:39.492327 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7e6ce3-2d65-4b41-acd4-b298b9f16440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:39.493875 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7e6ce3-2d65-4b41-acd4-b298b9f16440 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:39.494257 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7e6ce3-2d65-4b41-acd4-b298b9f16440 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:39.494575 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7e6ce3-2d65-4b41-acd4-b298b9f16440 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:39.495480 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7e6ce3-2d65-4b41-acd4-b298b9f16440 None admin] RBAC: Adding query filter params (role.id=3ca2dcbbd29d4a31a3ab78f2299bc239, scope.project.id=3be66087f3954ee4bac1dc53ea4d59ed, user.id=12a220316b144048ad50f0affd87fa43) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:29:39.495806 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7e6ce3-2d65-4b41-acd4-b298b9f16440 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:39.497579 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7e6ce3-2d65-4b41-acd4-b298b9f16440 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:39.509316 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 47/93] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jun 16 17:29:39 2022] GET /identity/v3/role_assignments?user.id=12a220316b144048ad50f0affd87fa43&scope.project.id=3be66087f3954ee4bac1dc53ea4d59ed&role.id=3ca2dcbbd29d4a31a3ab78f2299bc239 => generated 259 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:29:39.514193 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:39.518068 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7e6ce3-2d65-4b41-acd4-b298b9f16440 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:39.584728 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7519e00b-a5b5-4aeb-8bab-2ecff95f7a3d None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:39.636956 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd308ae7-edfc-4746-b3e1-d445fa6faf4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:39.637823 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd308ae7-edfc-4746-b3e1-d445fa6faf4a None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:39.638111 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd308ae7-edfc-4746-b3e1-d445fa6faf4a None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:39.638390 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd308ae7-edfc-4746-b3e1-d445fa6faf4a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:39.653665 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:39.689047 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7e6ce3-2d65-4b41-acd4-b298b9f16440 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:39.740063 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd308ae7-edfc-4746-b3e1-d445fa6faf4a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:39.741702 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd308ae7-edfc-4746-b3e1-d445fa6faf4a None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:39.742602 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd7ebd4-a246-4e69-a814-04c9a834db8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:39.743531 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd7ebd4-a246-4e69-a814-04c9a834db8e None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:39.744227 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd7ebd4-a246-4e69-a814-04c9a834db8e None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:39.744728 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd7ebd4-a246-4e69-a814-04c9a834db8e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:39.746740 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd7ebd4-a246-4e69-a814-04c9a834db8e None admin] RBAC: Adding query filter params (role.id=243fd0a11d94477986deb1dcfb3f31dd, scope.project.id=3be66087f3954ee4bac1dc53ea4d59ed, user.id=f72678162d55418190ac5579cb1eb41f) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:29:39.747292 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd7ebd4-a246-4e69-a814-04c9a834db8e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:39.749008 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd7ebd4-a246-4e69-a814-04c9a834db8e None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:39.757294 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 48/94] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jun 16 17:29:39 2022] GET /identity/v3/role_assignments?user.id=f72678162d55418190ac5579cb1eb41f&scope.project.id=3be66087f3954ee4bac1dc53ea4d59ed&role.id=243fd0a11d94477986deb1dcfb3f31dd => generated 259 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:29:39.820803 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:39.876893 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 47/95] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jun 16 17:29:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:29:39.882866 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd7ebd4-a246-4e69-a814-04c9a834db8e None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:40.008982 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd7ebd4-a246-4e69-a814-04c9a834db8e None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:40.055036 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1a8bb8-a299-46d0-9dee-512332468d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:40.056306 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1a8bb8-a299-46d0-9dee-512332468d69 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:40.056805 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1a8bb8-a299-46d0-9dee-512332468d69 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:40.057275 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1a8bb8-a299-46d0-9dee-512332468d69 None admin] PATH_INFO: `/v3/projects/afad0aa3b7c040bda29e672a36907ecb` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:40.058773 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1a8bb8-a299-46d0-9dee-512332468d69 None admin] RBAC: Authorizing `identity:get_project(project_id=afad0aa3b7c040bda29e672a36907ecb)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:40.060508 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1a8bb8-a299-46d0-9dee-512332468d69 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:40.062875 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 49/96] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jun 16 17:29:39 2022] GET /identity/v3/projects/afad0aa3b7c040bda29e672a36907ecb => generated 312 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:29:40.069638 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd308ae7-edfc-4746-b3e1-d445fa6faf4a None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:40.204465 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd308ae7-edfc-4746-b3e1-d445fa6faf4a None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:40.245785 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56178196-435b-45a8-a92c-12545eaf70f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:40.246560 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56178196-435b-45a8-a92c-12545eaf70f6 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:40.246852 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56178196-435b-45a8-a92c-12545eaf70f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:40.247181 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56178196-435b-45a8-a92c-12545eaf70f6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:40.248046 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56178196-435b-45a8-a92c-12545eaf70f6 None admin] RBAC: Adding query filter params (role.id=0be2075946f54c8485f1c275f746478a, scope.project.id=afad0aa3b7c040bda29e672a36907ecb, user.id=f72678162d55418190ac5579cb1eb41f) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:29:40.248435 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56178196-435b-45a8-a92c-12545eaf70f6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:40.249900 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56178196-435b-45a8-a92c-12545eaf70f6 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:40.261016 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 48/97] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jun 16 17:29:40 2022] GET /identity/v3/role_assignments?user.id=f72678162d55418190ac5579cb1eb41f&scope.project.id=afad0aa3b7c040bda29e672a36907ecb&role.id=0be2075946f54c8485f1c275f746478a => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:29:41.776170 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af61cd79-711e-4e1b-8a30-d2ff321af983 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:41.777324 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af61cd79-711e-4e1b-8a30-d2ff321af983 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:41.777830 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af61cd79-711e-4e1b-8a30-d2ff321af983 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:41.779423 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 50/98] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:29:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:29:41.785074 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb78fd9-471f-423f-8d60-a28b1baa84e0 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:41.785949 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb78fd9-471f-423f-8d60-a28b1baa84e0 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:41.786422 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb78fd9-471f-423f-8d60-a28b1baa84e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:41.808818 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-1cb78fd9-471f-423f-8d60-a28b1baa84e0 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:29:41.810351 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-1cb78fd9-471f-423f-8d60-a28b1baa84e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:29:41.840078 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:41.950353 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 49/99] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:29:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:29:41.985478 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af61cd79-711e-4e1b-8a30-d2ff321af983 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:42.004361 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:42.122331 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af61cd79-711e-4e1b-8a30-d2ff321af983 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:42.167465 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f601b4b1-4eff-400e-8e9c-6a832769c01c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:42.168274 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f601b4b1-4eff-400e-8e9c-6a832769c01c None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:42.168583 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f601b4b1-4eff-400e-8e9c-6a832769c01c None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:42.168862 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f601b4b1-4eff-400e-8e9c-6a832769c01c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:42.185881 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f601b4b1-4eff-400e-8e9c-6a832769c01c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:29:42.194304 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-f601b4b1-4eff-400e-8e9c-6a832769c01c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:29:42.194304 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:29:42.194304 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:29:42.194304 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:29:42.194304 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:29:42.194304 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:29:42.194304 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:29:42.194304 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:29:42.194304 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:29:42.194304 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:29:42.194304 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:29:42.194304 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:29:42.194304 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:29:42.194304 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:29:42.194304 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:29:42.194304 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:29:42.195238 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:29:42.195238 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:29:42.195238 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:29:42.195238 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:29:42.195238 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:29:42.195238 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:29:42.195238 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:29:42.195238 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:29:42.195238 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:29:42.195238 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:29:42.195238 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:29:42.195238 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:29:42.195238 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:29:42.195238 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:29:42.195238 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:29:42.195238 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:29:42.196016 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:29:42.196016 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:29:42.196016 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:29:42.196016 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:29:42.196016 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:29:42.196016 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:29:42.196016 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:29:42.196016 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:29:42.196016 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:29:42.196016 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:29:42.196016 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:29:42.196016 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:29:42.196016 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 51/100] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jun 16 17:29:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:29:42.203347 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb78fd9-471f-423f-8d60-a28b1baa84e0 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:42.338944 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb78fd9-471f-423f-8d60-a28b1baa84e0 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:42.387960 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c95249-2235-4613-a420-323c46c9a13a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:42.389023 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c95249-2235-4613-a420-323c46c9a13a None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:42.389518 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c95249-2235-4613-a420-323c46c9a13a None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:42.389991 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c95249-2235-4613-a420-323c46c9a13a None admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:42.391483 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c95249-2235-4613-a420-323c46c9a13a None admin] RBAC: Adding query filter params (name=member) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:29:42.392286 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c95249-2235-4613-a420-323c46c9a13a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:42.394188 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c95249-2235-4613-a420-323c46c9a13a None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:42.408958 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 50/101] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Jun 16 17:29:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:29:42.414812 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f601b4b1-4eff-400e-8e9c-6a832769c01c None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:42.508241 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:42.558422 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f601b4b1-4eff-400e-8e9c-6a832769c01c None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:42.600993 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115c29a1-8eef-4623-8188-6da99c0acc17 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:42.601932 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115c29a1-8eef-4623-8188-6da99c0acc17 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:42.602695 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115c29a1-8eef-4623-8188-6da99c0acc17 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:42.605741 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 51/102] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:29:42 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:29:42.610689 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd27692a-f13c-47cb-93f9-aefbea22a7f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:42.611592 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd27692a-f13c-47cb-93f9-aefbea22a7f6 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:42.611875 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd27692a-f13c-47cb-93f9-aefbea22a7f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:42.612150 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd27692a-f13c-47cb-93f9-aefbea22a7f6 None admin] PATH_INFO: `/v3/users/f72678162d55418190ac5579cb1eb41f` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:42.613207 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef84dac-f26c-492a-9b1a-5fd6f09c3631 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:42.613458 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd27692a-f13c-47cb-93f9-aefbea22a7f6 None admin] RBAC: Authorizing `identity:get_user(user_id=f72678162d55418190ac5579cb1eb41f)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:42.614084 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef84dac-f26c-492a-9b1a-5fd6f09c3631 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:42.614556 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef84dac-f26c-492a-9b1a-5fd6f09c3631 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:42.614918 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd27692a-f13c-47cb-93f9-aefbea22a7f6 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:42.616582 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 52/103] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jun 16 17:29:42 2022] GET /identity/v3/users/f72678162d55418190ac5579cb1eb41f => generated 275 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:29:42.626327 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd27692a-f13c-47cb-93f9-aefbea22a7f6 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:42.648064 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-4ef84dac-f26c-492a-9b1a-5fd6f09c3631 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:29:42.649661 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-4ef84dac-f26c-492a-9b1a-5fd6f09c3631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:29:42.665887 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:42.764221 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd27692a-f13c-47cb-93f9-aefbea22a7f6 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:42.777901 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 52/104] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:29:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:29:42.814088 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef84dac-f26c-492a-9b1a-5fd6f09c3631 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:42.823354 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e599c7-0d17-4ed8-a71e-ec500b458503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:42.824239 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e599c7-0d17-4ed8-a71e-ec500b458503 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:42.824555 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e599c7-0d17-4ed8-a71e-ec500b458503 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:42.824835 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e599c7-0d17-4ed8-a71e-ec500b458503 None admin] PATH_INFO: `/v3/projects/3be66087f3954ee4bac1dc53ea4d59ed` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:42.826336 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e599c7-0d17-4ed8-a71e-ec500b458503 None admin] RBAC: Authorizing `identity:get_project(project_id=3be66087f3954ee4bac1dc53ea4d59ed)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:42.827866 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e599c7-0d17-4ed8-a71e-ec500b458503 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:42.832400 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 53/105] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jun 16 17:29:42 2022] GET /identity/v3/projects/3be66087f3954ee4bac1dc53ea4d59ed => generated 298 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:29:42.838771 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e599c7-0d17-4ed8-a71e-ec500b458503 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:42.841485 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:42.979639 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e599c7-0d17-4ed8-a71e-ec500b458503 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:43.071360 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef84dac-f26c-492a-9b1a-5fd6f09c3631 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:43.076347 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f46297-a70a-4802-8929-24305104d8f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:43.077289 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f46297-a70a-4802-8929-24305104d8f2 None admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:43.077580 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f46297-a70a-4802-8929-24305104d8f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:43.077869 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f46297-a70a-4802-8929-24305104d8f2 None admin] PATH_INFO: `/v3/projects/3be66087f3954ee4bac1dc53ea4d59ed/users/f72678162d55418190ac5579cb1eb41f/roles/0be2075946f54c8485f1c275f746478a` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:43.081028 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f46297-a70a-4802-8929-24305104d8f2 None admin] RBAC: Authorizing `identity:create_grant(project_id=3be66087f3954ee4bac1dc53ea4d59ed, user_id=f72678162d55418190ac5579cb1eb41f, role_id=0be2075946f54c8485f1c275f746478a)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:43.082816 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f46297-a70a-4802-8929-24305104d8f2 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:43.098983 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 54/106] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Jun 16 17:29:42 2022] PUT /identity/v3/projects/3be66087f3954ee4bac1dc53ea4d59ed/users/f72678162d55418190ac5579cb1eb41f/roles/0be2075946f54c8485f1c275f746478a => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:29:43.100763 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497bc480-de51-46c7-b18f-a0dbaad15dc2 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:43.101319 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497bc480-de51-46c7-b18f-a0dbaad15dc2 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:43.101822 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497bc480-de51-46c7-b18f-a0dbaad15dc2 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:43.103541 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 55/107] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:29:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:29:43.105176 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052906ee-d60a-42ec-bc69-69c6ea2062de None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:43.105675 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052906ee-d60a-42ec-bc69-69c6ea2062de None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:43.106183 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052906ee-d60a-42ec-bc69-69c6ea2062de None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:43.108057 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 56/108] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:29:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:29:43.112755 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62fb0bb-64cd-4102-93b4-8989781f5c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:43.113493 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62fb0bb-64cd-4102-93b4-8989781f5c30 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:43.113833 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62fb0bb-64cd-4102-93b4-8989781f5c30 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:43.114308 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62fb0bb-64cd-4102-93b4-8989781f5c30 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:43.115205 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084d8edc-1336-476b-8019-ae9535f30bc0 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:43.115823 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084d8edc-1336-476b-8019-ae9535f30bc0 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:43.116391 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084d8edc-1336-476b-8019-ae9535f30bc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:43.129240 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62fb0bb-64cd-4102-93b4-8989781f5c30 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:29:43.137106 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-084d8edc-1336-476b-8019-ae9535f30bc0 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:29:43.137872 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-d62fb0bb-64cd-4102-93b4-8989781f5c30 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 17:29:43.137872 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:29:43.137872 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:29:43.137872 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:29:43.137872 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:29:43.137872 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:29:43.137872 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:29:43.137872 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:29:43.137872 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:29:43.137872 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:29:43.137872 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:29:43.137872 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:29:43.137872 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:29:43.137872 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:29:43.137872 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:29:43.137872 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:29:43.138749 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:29:43.138749 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:29:43.138749 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:29:43.138749 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:29:43.138749 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:29:43.138749 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:29:43.138749 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:29:43.138749 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:29:43.138749 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:29:43.138749 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:29:43.138749 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:29:43.138749 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:29:43.138749 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:29:43.138749 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:29:43.138749 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:29:43.138749 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:29:43.140002 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:29:43.140002 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:29:43.140002 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:29:43.140002 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:29:43.140002 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:29:43.140002 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:29:43.140002 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:29:43.140002 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:29:43.140002 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:29:43.140002 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:29:43.140002 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 17:29:43.140002 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:29:43.140002 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 53/109] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Jun 16 17:29:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:29:43.140666 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa5c1fc-aef6-4f74-9acd-59aa5048c197 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:43.141449 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa5c1fc-aef6-4f74-9acd-59aa5048c197 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:43.141449 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa5c1fc-aef6-4f74-9acd-59aa5048c197 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:43.143396 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-084d8edc-1336-476b-8019-ae9535f30bc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:29:43.153907 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:43.183188 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-2aa5c1fc-aef6-4f74-9acd-59aa5048c197 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:29:43.188733 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-2aa5c1fc-aef6-4f74-9acd-59aa5048c197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:29:43.252853 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:43.270274 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 57/110] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:29:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:29:43.273536 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084d8edc-1336-476b-8019-ae9535f30bc0 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:43.346306 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 54/111] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:29:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:29:43.347266 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa5c1fc-aef6-4f74-9acd-59aa5048c197 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:43.423683 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084d8edc-1336-476b-8019-ae9535f30bc0 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:43.469828 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02682875-b816-44bf-9756-34445b6a4c9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:43.470518 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02682875-b816-44bf-9756-34445b6a4c9e None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:43.470788 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa5c1fc-aef6-4f74-9acd-59aa5048c197 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:43.470917 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02682875-b816-44bf-9756-34445b6a4c9e None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:43.471069 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02682875-b816-44bf-9756-34445b6a4c9e None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:43.471824 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02682875-b816-44bf-9756-34445b6a4c9e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:29:43.472138 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02682875-b816-44bf-9756-34445b6a4c9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:43.473903 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02682875-b816-44bf-9756-34445b6a4c9e None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:43.484864 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 58/112] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jun 16 17:29:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:29:43.486881 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c02bb9-2280-4be0-891d-ef55e48870ad None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:43.487185 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c02bb9-2280-4be0-891d-ef55e48870ad None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:43.487474 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c02bb9-2280-4be0-891d-ef55e48870ad None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:43.489160 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 59/113] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:29:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:29:43.489943 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c02bb9-2280-4be0-891d-ef55e48870ad None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:43.519707 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39af509d-777d-4b92-8d42-f952dbc8e766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:43.521151 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39af509d-777d-4b92-8d42-f952dbc8e766 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:43.521716 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39af509d-777d-4b92-8d42-f952dbc8e766 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:43.522199 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39af509d-777d-4b92-8d42-f952dbc8e766 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:43.525450 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:43.539481 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39af509d-777d-4b92-8d42-f952dbc8e766 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:29:43.549812 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-39af509d-777d-4b92-8d42-f952dbc8e766 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 17:29:43.549812 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:29:43.549812 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:29:43.549812 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:29:43.549812 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:29:43.549812 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:29:43.549812 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:29:43.549812 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:29:43.549812 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:29:43.549812 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:29:43.549812 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:29:43.549812 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:29:43.549812 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:29:43.549812 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:29:43.549812 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:29:43.549812 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:29:43.550764 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:29:43.550764 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:29:43.550764 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:29:43.550764 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:29:43.550764 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:29:43.550764 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:29:43.550764 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:29:43.550764 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:29:43.550764 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:29:43.550764 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:29:43.550764 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:29:43.550764 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:29:43.550764 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:29:43.550764 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:29:43.550764 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:29:43.550764 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:29:43.551502 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:29:43.551502 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:29:43.551502 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:29:43.551502 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:29:43.551502 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:29:43.551502 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:29:43.551502 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:29:43.551502 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:29:43.551502 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:29:43.551502 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:29:43.551502 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 17:29:43.551502 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:29:43.553895 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 55/114] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jun 16 17:29:43 2022] GET /identity/v3/roles/reader => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:29:43.559184 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4827dab-fa47-469b-bcad-c238cf6c43d3 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:43.559184 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4827dab-fa47-469b-bcad-c238cf6c43d3 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:43.559184 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4827dab-fa47-469b-bcad-c238cf6c43d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:43.578167 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:43.582621 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-c4827dab-fa47-469b-bcad-c238cf6c43d3 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:29:43.583829 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-c4827dab-fa47-469b-bcad-c238cf6c43d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:29:43.631413 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c02bb9-2280-4be0-891d-ef55e48870ad None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:43.667162 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 56/115] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:29:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:29:43.668393 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4827dab-fa47-469b-bcad-c238cf6c43d3 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:43.687534 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5694e23-723a-4933-aff6-17f31e7d5766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:43.689761 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5694e23-723a-4933-aff6-17f31e7d5766 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:43.692246 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5694e23-723a-4933-aff6-17f31e7d5766 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:43.692829 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5694e23-723a-4933-aff6-17f31e7d5766 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:43.694535 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5694e23-723a-4933-aff6-17f31e7d5766 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:43.696318 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5694e23-723a-4933-aff6-17f31e7d5766 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:43.699806 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 60/116] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jun 16 17:29:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:29:43.701679 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5694e23-723a-4933-aff6-17f31e7d5766 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:43.798754 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4827dab-fa47-469b-bcad-c238cf6c43d3 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:43.819131 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5694e23-723a-4933-aff6-17f31e7d5766 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:43.824656 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:43.838686 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedcef7c-04b3-4523-880c-d4709cd765db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:43.839573 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedcef7c-04b3-4523-880c-d4709cd765db None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:43.839913 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedcef7c-04b3-4523-880c-d4709cd765db None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:43.840196 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedcef7c-04b3-4523-880c-d4709cd765db None admin] PATH_INFO: `/v3/users/f72678162d55418190ac5579cb1eb41f` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:43.841447 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedcef7c-04b3-4523-880c-d4709cd765db None admin] RBAC: Authorizing `identity:get_user(user_id=f72678162d55418190ac5579cb1eb41f)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:43.842923 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedcef7c-04b3-4523-880c-d4709cd765db None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:43.845345 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 57/117] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jun 16 17:29:43 2022] GET /identity/v3/users/f72678162d55418190ac5579cb1eb41f => generated 275 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:29:43.846064 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedcef7c-04b3-4523-880c-d4709cd765db None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:43.852828 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:43.866045 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1794ebd0-faef-45c4-8a63-851d1bae13cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:43.867081 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1794ebd0-faef-45c4-8a63-851d1bae13cf None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:43.867634 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1794ebd0-faef-45c4-8a63-851d1bae13cf None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:43.868112 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1794ebd0-faef-45c4-8a63-851d1bae13cf None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:43.869210 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1794ebd0-faef-45c4-8a63-851d1bae13cf None admin] RBAC: Adding query filter params (name=reader) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:29:43.869765 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1794ebd0-faef-45c4-8a63-851d1bae13cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:43.871351 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1794ebd0-faef-45c4-8a63-851d1bae13cf None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:43.892836 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 61/118] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Jun 16 17:29:43 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:29:43.894202 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1794ebd0-faef-45c4-8a63-851d1bae13cf None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:43.954238 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedcef7c-04b3-4523-880c-d4709cd765db None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:43.998722 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59be8ab0-11a8-46a9-978c-91db64bc238c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:43.999633 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59be8ab0-11a8-46a9-978c-91db64bc238c None admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:43.999929 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59be8ab0-11a8-46a9-978c-91db64bc238c None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:44.000291 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59be8ab0-11a8-46a9-978c-91db64bc238c None admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:44.002054 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59be8ab0-11a8-46a9-978c-91db64bc238c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:44.003802 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59be8ab0-11a8-46a9-978c-91db64bc238c None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:44.004351 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1794ebd0-faef-45c4-8a63-851d1bae13cf None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:44.040951 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 58/119] 10.222.0.42 () {60 vars in 1141 bytes} [Thu Jun 16 17:29:43 2022] POST /identity/v3/projects => generated 302 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:29:44.043447 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7c69ce-6d09-4502-9d0a-7744c306ee65 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:44.044191 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7c69ce-6d09-4502-9d0a-7744c306ee65 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:44.044191 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7c69ce-6d09-4502-9d0a-7744c306ee65 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:44.050414 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 59/120] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:29:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:29:44.050414 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7c69ce-6d09-4502-9d0a-7744c306ee65 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:44.053713 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1625817-225d-4b53-825c-fd92225868bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:44.054790 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1625817-225d-4b53-825c-fd92225868bf None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:44.055277 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1625817-225d-4b53-825c-fd92225868bf None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:44.055730 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1625817-225d-4b53-825c-fd92225868bf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:44.070317 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1625817-225d-4b53-825c-fd92225868bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:29:44.080159 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-c1625817-225d-4b53-825c-fd92225868bf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:29:44.080159 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:29:44.080159 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:29:44.080159 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:29:44.080159 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:29:44.080159 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:29:44.080159 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:29:44.080159 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:29:44.080159 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:29:44.080159 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:29:44.080159 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:29:44.080159 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:29:44.080159 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:29:44.080159 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:29:44.080159 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:29:44.080159 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:29:44.081138 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:29:44.081138 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:29:44.081138 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:29:44.081138 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:29:44.081138 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:29:44.081138 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:29:44.081138 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:29:44.081138 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:29:44.081138 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:29:44.081138 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:29:44.081138 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:29:44.081138 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:29:44.081138 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:29:44.081138 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:29:44.081138 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:29:44.081138 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:29:44.081920 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:29:44.081920 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:29:44.081920 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:29:44.081920 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:29:44.081920 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:29:44.081920 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:29:44.081920 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:29:44.081920 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:29:44.081920 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:29:44.081920 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:29:44.081920 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:29:44.081920 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:29:44.089868 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 62/121] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Jun 16 17:29:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:29:44.091259 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1625817-225d-4b53-825c-fd92225868bf None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:44.136329 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:44.162227 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:44.203802 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7c69ce-6d09-4502-9d0a-7744c306ee65 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:44.233571 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1625817-225d-4b53-825c-fd92225868bf None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:44.246366 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5467793f-d682-42cc-9e18-8c9342df6391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:44.247579 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5467793f-d682-42cc-9e18-8c9342df6391 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:44.247864 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5467793f-d682-42cc-9e18-8c9342df6391 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:44.248141 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5467793f-d682-42cc-9e18-8c9342df6391 None admin] PATH_INFO: `/v3/projects/3be66087f3954ee4bac1dc53ea4d59ed` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:44.249506 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5467793f-d682-42cc-9e18-8c9342df6391 None admin] RBAC: Authorizing `identity:get_project(project_id=3be66087f3954ee4bac1dc53ea4d59ed)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:44.251563 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5467793f-d682-42cc-9e18-8c9342df6391 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:44.254314 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 60/122] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jun 16 17:29:44 2022] GET /identity/v3/projects/3be66087f3954ee4bac1dc53ea4d59ed => generated 298 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:29:44.260126 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6babbf-b80f-4d90-8388-312cc4cfa010 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:44.260486 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6babbf-b80f-4d90-8388-312cc4cfa010 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:44.260998 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6babbf-b80f-4d90-8388-312cc4cfa010 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:44.279464 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2c79e6-bba4-41bf-a340-9e0ce7543299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:44.279947 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-4e6babbf-b80f-4d90-8388-312cc4cfa010 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:29:44.280342 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2c79e6-bba4-41bf-a340-9e0ce7543299 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:44.280658 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2c79e6-bba4-41bf-a340-9e0ce7543299 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:44.280935 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2c79e6-bba4-41bf-a340-9e0ce7543299 None admin] PATH_INFO: `/v3/users/ebd6465c9b4f4f4aaed32bb07032d7f0` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:44.281187 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-4e6babbf-b80f-4d90-8388-312cc4cfa010 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:29:44.303910 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2c79e6-bba4-41bf-a340-9e0ce7543299 None admin] RBAC: Authorizing `identity:get_user(user_id=ebd6465c9b4f4f4aaed32bb07032d7f0)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:44.305954 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2c79e6-bba4-41bf-a340-9e0ce7543299 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:44.308680 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 63/123] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jun 16 17:29:44 2022] GET /identity/v3/users/ebd6465c9b4f4f4aaed32bb07032d7f0 => generated 289 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:29:44.310285 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2c79e6-bba4-41bf-a340-9e0ce7543299 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:44.362227 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 61/124] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:29:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:29:44.362900 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6babbf-b80f-4d90-8388-312cc4cfa010 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:44.426876 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:44.447685 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2c79e6-bba4-41bf-a340-9e0ce7543299 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:44.450507 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:44.489209 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6babbf-b80f-4d90-8388-312cc4cfa010 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:44.495548 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cba1106-5b89-4f50-a84f-ea1084d4e474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:44.496494 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cba1106-5b89-4f50-a84f-ea1084d4e474 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:44.496921 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cba1106-5b89-4f50-a84f-ea1084d4e474 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:44.497826 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cba1106-5b89-4f50-a84f-ea1084d4e474 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:44.498872 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cba1106-5b89-4f50-a84f-ea1084d4e474 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:29:44.499343 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cba1106-5b89-4f50-a84f-ea1084d4e474 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:44.502198 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cba1106-5b89-4f50-a84f-ea1084d4e474 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:44.513068 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 64/125] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jun 16 17:29:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:29:44.517085 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cba1106-5b89-4f50-a84f-ea1084d4e474 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:44.534185 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2777a260-ab74-4e86-a661-79ff0485d904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:44.535281 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2777a260-ab74-4e86-a661-79ff0485d904 None admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:44.535659 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2777a260-ab74-4e86-a661-79ff0485d904 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:44.536063 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2777a260-ab74-4e86-a661-79ff0485d904 None admin] PATH_INFO: `/v3/projects/3be66087f3954ee4bac1dc53ea4d59ed/users/f72678162d55418190ac5579cb1eb41f/roles/243fd0a11d94477986deb1dcfb3f31dd` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:44.538562 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2777a260-ab74-4e86-a661-79ff0485d904 None admin] RBAC: Authorizing `identity:create_grant(project_id=3be66087f3954ee4bac1dc53ea4d59ed, user_id=f72678162d55418190ac5579cb1eb41f, role_id=243fd0a11d94477986deb1dcfb3f31dd)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:44.540146 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2777a260-ab74-4e86-a661-79ff0485d904 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:44.556884 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 62/126] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Jun 16 17:29:44 2022] PUT /identity/v3/projects/3be66087f3954ee4bac1dc53ea4d59ed/users/f72678162d55418190ac5579cb1eb41f/roles/243fd0a11d94477986deb1dcfb3f31dd => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:29:44.565414 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2777a260-ab74-4e86-a661-79ff0485d904 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:44.732000 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cba1106-5b89-4f50-a84f-ea1084d4e474 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:44.786954 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2777a260-ab74-4e86-a661-79ff0485d904 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:44.792254 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:44.797977 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a6ecfc-edff-48cc-b53e-1786efe82c3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:44.799058 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a6ecfc-edff-48cc-b53e-1786efe82c3a None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:44.799362 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a6ecfc-edff-48cc-b53e-1786efe82c3a None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:44.799602 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a6ecfc-edff-48cc-b53e-1786efe82c3a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:44.849293 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659d0f14-d3f2-4efb-9319-2472caee8689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:44.850518 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659d0f14-d3f2-4efb-9319-2472caee8689 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:44.850960 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659d0f14-d3f2-4efb-9319-2472caee8689 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:44.851369 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659d0f14-d3f2-4efb-9319-2472caee8689 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:44.852328 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:44.864250 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659d0f14-d3f2-4efb-9319-2472caee8689 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:29:44.872202 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-659d0f14-d3f2-4efb-9319-2472caee8689 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:29:44.872202 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:29:44.872202 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:29:44.872202 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:29:44.872202 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:29:44.872202 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:29:44.872202 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:29:44.872202 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:29:44.872202 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:29:44.872202 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:29:44.872202 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:29:44.872202 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:29:44.872202 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:29:44.872202 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:29:44.872202 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:29:44.872202 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:29:44.873181 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:29:44.873181 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:29:44.873181 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:29:44.873181 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:29:44.873181 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:29:44.873181 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:29:44.873181 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:29:44.873181 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:29:44.873181 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:29:44.873181 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:29:44.873181 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:29:44.873181 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:29:44.873181 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:29:44.873181 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:29:44.873181 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:29:44.873181 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:29:44.874158 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:29:44.874158 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:29:44.874158 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:29:44.874158 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:29:44.874158 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:29:44.874158 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:29:44.874158 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:29:44.874158 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:29:44.874158 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:29:44.874158 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:29:44.874158 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:29:44.874158 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:29:44.874935 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 63/127] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jun 16 17:29:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:29:44.876010 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659d0f14-d3f2-4efb-9319-2472caee8689 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:44.898965 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a6ecfc-edff-48cc-b53e-1786efe82c3a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:44.900496 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a6ecfc-edff-48cc-b53e-1786efe82c3a None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:45.050495 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659d0f14-d3f2-4efb-9319-2472caee8689 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:45.074171 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 65/128] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jun 16 17:29:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 561 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:29:45.074833 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a6ecfc-edff-48cc-b53e-1786efe82c3a None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:45.116055 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58ebdb1-cc05-416f-a3dd-6222a2a1fb8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:45.116887 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58ebdb1-cc05-416f-a3dd-6222a2a1fb8d None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:45.117169 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58ebdb1-cc05-416f-a3dd-6222a2a1fb8d None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:45.117447 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58ebdb1-cc05-416f-a3dd-6222a2a1fb8d None admin] PATH_INFO: `/v3/users/12a220316b144048ad50f0affd87fa43` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:45.119318 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58ebdb1-cc05-416f-a3dd-6222a2a1fb8d None admin] RBAC: Authorizing `identity:get_user(user_id=12a220316b144048ad50f0affd87fa43)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:45.120814 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58ebdb1-cc05-416f-a3dd-6222a2a1fb8d None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:45.124547 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 64/129] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jun 16 17:29:44 2022] GET /identity/v3/users/12a220316b144048ad50f0affd87fa43 => generated 247 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:29:45.126250 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58ebdb1-cc05-416f-a3dd-6222a2a1fb8d None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:45.173021 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:45.213268 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:45.275398 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a6ecfc-edff-48cc-b53e-1786efe82c3a None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:45.323497 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58ebdb1-cc05-416f-a3dd-6222a2a1fb8d None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:45.341243 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9481dc-148b-45f8-99f0-700d2cd22ced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:45.342993 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9481dc-148b-45f8-99f0-700d2cd22ced None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:45.343633 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9481dc-148b-45f8-99f0-700d2cd22ced None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:45.345953 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9481dc-148b-45f8-99f0-700d2cd22ced None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:45.346962 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9481dc-148b-45f8-99f0-700d2cd22ced None admin] RBAC: Adding query filter params (name=member) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:29:45.347428 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9481dc-148b-45f8-99f0-700d2cd22ced None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:45.349177 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9481dc-148b-45f8-99f0-700d2cd22ced None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:45.361941 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 66/130] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Jun 16 17:29:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:29:45.363102 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9481dc-148b-45f8-99f0-700d2cd22ced None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:45.394788 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e99545-c6c9-469b-ba40-6ff94cd9b6fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:45.395967 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e99545-c6c9-469b-ba40-6ff94cd9b6fe None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:45.396409 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e99545-c6c9-469b-ba40-6ff94cd9b6fe None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:45.396821 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e99545-c6c9-469b-ba40-6ff94cd9b6fe None admin] PATH_INFO: `/v3/projects/3be66087f3954ee4bac1dc53ea4d59ed` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:45.399121 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e99545-c6c9-469b-ba40-6ff94cd9b6fe None admin] RBAC: Authorizing `identity:get_project(project_id=3be66087f3954ee4bac1dc53ea4d59ed)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:45.401362 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e99545-c6c9-469b-ba40-6ff94cd9b6fe None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:45.405939 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 65/131] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jun 16 17:29:45 2022] GET /identity/v3/projects/3be66087f3954ee4bac1dc53ea4d59ed => generated 298 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:29:45.408013 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e99545-c6c9-469b-ba40-6ff94cd9b6fe None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:45.571807 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:45.580418 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e99545-c6c9-469b-ba40-6ff94cd9b6fe None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:45.581464 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9481dc-148b-45f8-99f0-700d2cd22ced None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:45.647924 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:45.660878 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e41ad3-e590-498d-8b0e-20b893e9368c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:45.661767 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e41ad3-e590-498d-8b0e-20b893e9368c None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:45.662066 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e41ad3-e590-498d-8b0e-20b893e9368c None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:45.662350 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e41ad3-e590-498d-8b0e-20b893e9368c None admin] PATH_INFO: `/v3/projects/3be66087f3954ee4bac1dc53ea4d59ed` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:45.663567 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1816de4-9307-4fda-b1ec-1107481f2de5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:45.664474 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1816de4-9307-4fda-b1ec-1107481f2de5 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:45.664762 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1816de4-9307-4fda-b1ec-1107481f2de5 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:45.665042 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1816de4-9307-4fda-b1ec-1107481f2de5 None admin] PATH_INFO: `/v3/users/f72678162d55418190ac5579cb1eb41f` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:45.666253 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e41ad3-e590-498d-8b0e-20b893e9368c None admin] RBAC: Authorizing `identity:get_project(project_id=3be66087f3954ee4bac1dc53ea4d59ed)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:45.667728 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e41ad3-e590-498d-8b0e-20b893e9368c None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:45.668658 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1816de4-9307-4fda-b1ec-1107481f2de5 None admin] RBAC: Authorizing `identity:get_user(user_id=f72678162d55418190ac5579cb1eb41f)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:45.670163 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1816de4-9307-4fda-b1ec-1107481f2de5 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:45.672186 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 66/132] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jun 16 17:29:45 2022] GET /identity/v3/users/f72678162d55418190ac5579cb1eb41f => generated 275 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:29:45.673548 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1816de4-9307-4fda-b1ec-1107481f2de5 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:45.673914 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 67/133] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jun 16 17:29:45 2022] GET /identity/v3/projects/3be66087f3954ee4bac1dc53ea4d59ed => generated 298 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:29:45.675283 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dcd4d4-9580-4db6-8070-63aaccc94a44 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:45.675580 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dcd4d4-9580-4db6-8070-63aaccc94a44 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:45.675867 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dcd4d4-9580-4db6-8070-63aaccc94a44 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:45.680851 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 68/134] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:29:45 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:29:45.683717 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dcd4d4-9580-4db6-8070-63aaccc94a44 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:45.794844 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1816de4-9307-4fda-b1ec-1107481f2de5 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:45.817112 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dcd4d4-9580-4db6-8070-63aaccc94a44 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:45.832273 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6677295-9073-4a97-a07b-a987c24dff02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:45.832990 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6677295-9073-4a97-a07b-a987c24dff02 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:45.833282 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6677295-9073-4a97-a07b-a987c24dff02 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:45.833567 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6677295-9073-4a97-a07b-a987c24dff02 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:45.834804 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6677295-9073-4a97-a07b-a987c24dff02 None admin] RBAC: Adding query filter params (role.id=9ed5382e55d649faa9195c4a9eb6fa57, scope.project.id=3be66087f3954ee4bac1dc53ea4d59ed, user.id=ebd6465c9b4f4f4aaed32bb07032d7f0) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:29:45.835314 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6677295-9073-4a97-a07b-a987c24dff02 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:45.837125 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6677295-9073-4a97-a07b-a987c24dff02 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:45.848431 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 67/135] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jun 16 17:29:45 2022] GET /identity/v3/role_assignments?user.id=ebd6465c9b4f4f4aaed32bb07032d7f0&scope.project.id=3be66087f3954ee4bac1dc53ea4d59ed&role.id=9ed5382e55d649faa9195c4a9eb6fa57 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:29:45.848431 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6677295-9073-4a97-a07b-a987c24dff02 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:45.884299 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:45.897251 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b961e7-c861-4ea5-8cc8-e00fbed98823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:45.898245 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b961e7-c861-4ea5-8cc8-e00fbed98823 None admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:45.898474 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b961e7-c861-4ea5-8cc8-e00fbed98823 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:45.898755 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b961e7-c861-4ea5-8cc8-e00fbed98823 None admin] PATH_INFO: `/v3/projects/3be66087f3954ee4bac1dc53ea4d59ed/users/12a220316b144048ad50f0affd87fa43/roles/3ca2dcbbd29d4a31a3ab78f2299bc239` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:45.906006 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b961e7-c861-4ea5-8cc8-e00fbed98823 None admin] RBAC: Authorizing `identity:create_grant(project_id=3be66087f3954ee4bac1dc53ea4d59ed, user_id=12a220316b144048ad50f0affd87fa43, role_id=3ca2dcbbd29d4a31a3ab78f2299bc239)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:45.907555 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b961e7-c861-4ea5-8cc8-e00fbed98823 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:45.947692 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 69/136] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Jun 16 17:29:45 2022] PUT /identity/v3/projects/3be66087f3954ee4bac1dc53ea4d59ed/users/12a220316b144048ad50f0affd87fa43/roles/3ca2dcbbd29d4a31a3ab78f2299bc239 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:29:45.949458 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2d4c61-154d-4f4d-b47a-4d6e5b235c71 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:45.950007 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2d4c61-154d-4f4d-b47a-4d6e5b235c71 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:45.950532 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2d4c61-154d-4f4d-b47a-4d6e5b235c71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:45.991223 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-ed2d4c61-154d-4f4d-b47a-4d6e5b235c71 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:29:45.993137 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-ed2d4c61-154d-4f4d-b47a-4d6e5b235c71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:29:46.048651 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6677295-9073-4a97-a07b-a987c24dff02 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:46.117201 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1246d7d0-3cea-4fee-b823-e1b9f5ffde57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:46.118498 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1246d7d0-3cea-4fee-b823-e1b9f5ffde57 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:46.118984 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1246d7d0-3cea-4fee-b823-e1b9f5ffde57 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:46.119458 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1246d7d0-3cea-4fee-b823-e1b9f5ffde57 None admin] PATH_INFO: `/v3/projects/afad0aa3b7c040bda29e672a36907ecb` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:46.120837 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:46.121458 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1246d7d0-3cea-4fee-b823-e1b9f5ffde57 None admin] RBAC: Authorizing `identity:get_project(project_id=afad0aa3b7c040bda29e672a36907ecb)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:46.122928 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1246d7d0-3cea-4fee-b823-e1b9f5ffde57 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:46.124713 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 68/137] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jun 16 17:29:45 2022] GET /identity/v3/projects/afad0aa3b7c040bda29e672a36907ecb => generated 312 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:29:46.141522 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 70/138] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:29:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:29:46.142706 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1246d7d0-3cea-4fee-b823-e1b9f5ffde57 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:46.169077 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2d4c61-154d-4f4d-b47a-4d6e5b235c71 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:46.366929 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:46.372177 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1246d7d0-3cea-4fee-b823-e1b9f5ffde57 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:46.383478 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2d4c61-154d-4f4d-b47a-4d6e5b235c71 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:46.412057 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5414b5-9d64-4a1d-9786-b2aab7ae7c3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:46.413440 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5414b5-9d64-4a1d-9786-b2aab7ae7c3e None admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:46.413735 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5414b5-9d64-4a1d-9786-b2aab7ae7c3e None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:46.414012 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5414b5-9d64-4a1d-9786-b2aab7ae7c3e None admin] PATH_INFO: `/v3/projects/afad0aa3b7c040bda29e672a36907ecb/users/f72678162d55418190ac5579cb1eb41f/roles/0be2075946f54c8485f1c275f746478a` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:46.416128 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5414b5-9d64-4a1d-9786-b2aab7ae7c3e None admin] RBAC: Authorizing `identity:create_grant(project_id=afad0aa3b7c040bda29e672a36907ecb, user_id=f72678162d55418190ac5579cb1eb41f, role_id=0be2075946f54c8485f1c275f746478a)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:46.417943 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5414b5-9d64-4a1d-9786-b2aab7ae7c3e None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:46.437205 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 69/139] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Jun 16 17:29:46 2022] PUT /identity/v3/projects/afad0aa3b7c040bda29e672a36907ecb/users/f72678162d55418190ac5579cb1eb41f/roles/0be2075946f54c8485f1c275f746478a => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:29:46.477858 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668b39a0-eac7-4f0a-acfc-739704404a28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:46.478708 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668b39a0-eac7-4f0a-acfc-739704404a28 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:46.479004 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668b39a0-eac7-4f0a-acfc-739704404a28 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:46.479295 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668b39a0-eac7-4f0a-acfc-739704404a28 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:46.501837 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668b39a0-eac7-4f0a-acfc-739704404a28 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:29:46.509862 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-668b39a0-eac7-4f0a-acfc-739704404a28 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:29:46.509862 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:29:46.509862 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:29:46.509862 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:29:46.509862 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:29:46.509862 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:29:46.509862 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:29:46.509862 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:29:46.509862 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:29:46.509862 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:29:46.509862 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:29:46.509862 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:29:46.509862 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:29:46.509862 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:29:46.509862 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:29:46.509862 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:29:46.510808 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:29:46.510808 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:29:46.510808 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:29:46.510808 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:29:46.510808 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:29:46.510808 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:29:46.510808 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:29:46.510808 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:29:46.510808 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:29:46.510808 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:29:46.510808 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:29:46.510808 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:29:46.510808 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:29:46.510808 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:29:46.510808 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:29:46.510808 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:29:46.511586 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:29:46.511586 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:29:46.511586 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:29:46.511586 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:29:46.511586 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:29:46.511586 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:29:46.511586 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:29:46.511586 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:29:46.511586 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:29:46.511586 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:29:46.511586 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:29:46.511586 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:29:46.511586 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 71/140] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jun 16 17:29:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:29:46.518441 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5414b5-9d64-4a1d-9786-b2aab7ae7c3e None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:46.660006 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5414b5-9d64-4a1d-9786-b2aab7ae7c3e None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:46.699712 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9d4c28-5267-4a96-b63b-540335bd581e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:46.700318 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9d4c28-5267-4a96-b63b-540335bd581e None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:46.700709 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9d4c28-5267-4a96-b63b-540335bd581e None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:46.700988 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9d4c28-5267-4a96-b63b-540335bd581e None admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:46.701763 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9d4c28-5267-4a96-b63b-540335bd581e None admin] RBAC: Adding query filter params (name=member) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:29:46.702070 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9d4c28-5267-4a96-b63b-540335bd581e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:46.703547 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9d4c28-5267-4a96-b63b-540335bd581e None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:46.717828 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 70/141] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Jun 16 17:29:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:29:46.725908 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668b39a0-eac7-4f0a-acfc-739704404a28 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:46.820503 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:46.892332 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668b39a0-eac7-4f0a-acfc-739704404a28 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:46.940786 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87dfad5-658f-4df5-80ca-19bbb6e1721e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:46.941679 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87dfad5-658f-4df5-80ca-19bbb6e1721e None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:46.941965 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87dfad5-658f-4df5-80ca-19bbb6e1721e None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:46.942242 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87dfad5-658f-4df5-80ca-19bbb6e1721e None admin] PATH_INFO: `/v3/users/f72678162d55418190ac5579cb1eb41f` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:46.945666 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87dfad5-658f-4df5-80ca-19bbb6e1721e None admin] RBAC: Authorizing `identity:get_user(user_id=f72678162d55418190ac5579cb1eb41f)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:46.947157 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87dfad5-658f-4df5-80ca-19bbb6e1721e None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:46.948888 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 72/142] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jun 16 17:29:46 2022] GET /identity/v3/users/f72678162d55418190ac5579cb1eb41f => generated 275 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:29:46.954922 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9d4c28-5267-4a96-b63b-540335bd581e None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:46.989060 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:47.100614 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9d4c28-5267-4a96-b63b-540335bd581e None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:47.143779 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7ea7a3-d6a4-4ad3-9c6c-5a301b5f6149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:47.144483 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7ea7a3-d6a4-4ad3-9c6c-5a301b5f6149 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:47.144771 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7ea7a3-d6a4-4ad3-9c6c-5a301b5f6149 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:47.145048 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7ea7a3-d6a4-4ad3-9c6c-5a301b5f6149 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:47.237106 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7ea7a3-d6a4-4ad3-9c6c-5a301b5f6149 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:47.238498 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7ea7a3-d6a4-4ad3-9c6c-5a301b5f6149 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:47.260779 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:47.266549 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cc4994-46eb-4ffd-bd79-6cb155133660 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:47.267372 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cc4994-46eb-4ffd-bd79-6cb155133660 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:47.267372 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cc4994-46eb-4ffd-bd79-6cb155133660 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:47.268793 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 73/143] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:29:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:29:47.285871 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b51b03-6a4d-4889-92fe-65cc2ad10e19 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:47.286154 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b51b03-6a4d-4889-92fe-65cc2ad10e19 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:47.286439 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b51b03-6a4d-4889-92fe-65cc2ad10e19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:47.341455 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-91b51b03-6a4d-4889-92fe-65cc2ad10e19 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:29:47.342670 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-91b51b03-6a4d-4889-92fe-65cc2ad10e19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:29:47.374827 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 71/144] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jun 16 17:29:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:29:47.377583 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7ea7a3-d6a4-4ad3-9c6c-5a301b5f6149 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:47.527428 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 74/145] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:29:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:29:47.530703 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd681dc-77b1-43d5-a1c1-3dd0f6d1493a None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:47.530995 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd681dc-77b1-43d5-a1c1-3dd0f6d1493a None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:47.531284 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd681dc-77b1-43d5-a1c1-3dd0f6d1493a None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:47.537420 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 75/146] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:29:47 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:29:47.538592 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a618532-89ef-4b9e-9ec9-b06ae8acafc8 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:47.538874 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a618532-89ef-4b9e-9ec9-b06ae8acafc8 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:47.539146 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a618532-89ef-4b9e-9ec9-b06ae8acafc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:47.581775 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-2a618532-89ef-4b9e-9ec9-b06ae8acafc8 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:29:47.583056 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-2a618532-89ef-4b9e-9ec9-b06ae8acafc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:29:47.608207 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7ea7a3-d6a4-4ad3-9c6c-5a301b5f6149 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:47.631543 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:47.650331 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15575d84-4594-4829-bc8f-7d39b92d8faf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:47.651483 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15575d84-4594-4829-bc8f-7d39b92d8faf None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:47.651767 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15575d84-4594-4829-bc8f-7d39b92d8faf None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:47.652035 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15575d84-4594-4829-bc8f-7d39b92d8faf None admin] PATH_INFO: `/v3/projects/3be66087f3954ee4bac1dc53ea4d59ed` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:47.654398 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15575d84-4594-4829-bc8f-7d39b92d8faf None admin] RBAC: Authorizing `identity:get_project(project_id=3be66087f3954ee4bac1dc53ea4d59ed)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:47.655848 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15575d84-4594-4829-bc8f-7d39b92d8faf None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:47.658356 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 72/147] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jun 16 17:29:47 2022] GET /identity/v3/projects/3be66087f3954ee4bac1dc53ea4d59ed => generated 298 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:29:47.663595 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15575d84-4594-4829-bc8f-7d39b92d8faf None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:47.673473 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:47.728665 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 76/148] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:29:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:29:47.729426 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a618532-89ef-4b9e-9ec9-b06ae8acafc8 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:47.888245 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a618532-89ef-4b9e-9ec9-b06ae8acafc8 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:47.910318 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15575d84-4594-4829-bc8f-7d39b92d8faf None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:47.934465 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1768f39-ce4f-4708-b17d-87953cea369e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:47.935144 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1768f39-ce4f-4708-b17d-87953cea369e None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:47.935433 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1768f39-ce4f-4708-b17d-87953cea369e None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:47.935713 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1768f39-ce4f-4708-b17d-87953cea369e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:47.936582 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1768f39-ce4f-4708-b17d-87953cea369e None admin] RBAC: Adding query filter params (role.id=0be2075946f54c8485f1c275f746478a, scope.project.id=3be66087f3954ee4bac1dc53ea4d59ed, user.id=f72678162d55418190ac5579cb1eb41f) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:29:47.936905 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1768f39-ce4f-4708-b17d-87953cea369e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:47.938925 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1768f39-ce4f-4708-b17d-87953cea369e None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:47.954176 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 77/149] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jun 16 17:29:47 2022] GET /identity/v3/role_assignments?user.id=f72678162d55418190ac5579cb1eb41f&scope.project.id=3be66087f3954ee4bac1dc53ea4d59ed&role.id=0be2075946f54c8485f1c275f746478a => generated 608 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:29:47.955028 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1768f39-ce4f-4708-b17d-87953cea369e None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:47.977881 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf65d5c-d40e-4cf0-b901-f56f75ffdd59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:47.978716 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf65d5c-d40e-4cf0-b901-f56f75ffdd59 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:47.979003 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf65d5c-d40e-4cf0-b901-f56f75ffdd59 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:47.979280 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf65d5c-d40e-4cf0-b901-f56f75ffdd59 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:47.981376 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf65d5c-d40e-4cf0-b901-f56f75ffdd59 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:47.983360 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf65d5c-d40e-4cf0-b901-f56f75ffdd59 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:47.986209 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 73/150] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jun 16 17:29:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:29:47.994604 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf65d5c-d40e-4cf0-b901-f56f75ffdd59 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:48.069837 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:48.112224 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:48.172874 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf65d5c-d40e-4cf0-b901-f56f75ffdd59 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:48.238983 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69644da7-7823-4905-a752-d9a98fcea778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:48.239671 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69644da7-7823-4905-a752-d9a98fcea778 None admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:48.239959 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69644da7-7823-4905-a752-d9a98fcea778 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:48.240236 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69644da7-7823-4905-a752-d9a98fcea778 None admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:48.240997 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69644da7-7823-4905-a752-d9a98fcea778 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:48.242485 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69644da7-7823-4905-a752-d9a98fcea778 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:48.268417 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1768f39-ce4f-4708-b17d-87953cea369e None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:48.285389 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 74/151] 10.222.0.42 () {60 vars in 1132 bytes} [Thu Jun 16 17:29:47 2022] POST /identity/v3/users => generated 283 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:29:48.315752 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84df9786-21e6-4649-a16a-d52778306f6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:48.316637 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84df9786-21e6-4649-a16a-d52778306f6c None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:48.317043 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84df9786-21e6-4649-a16a-d52778306f6c None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:48.317214 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84df9786-21e6-4649-a16a-d52778306f6c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:48.330859 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84df9786-21e6-4649-a16a-d52778306f6c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:29:48.339323 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-84df9786-21e6-4649-a16a-d52778306f6c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 17:29:48.339323 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:29:48.339323 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:29:48.339323 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:29:48.339323 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:29:48.339323 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:29:48.339323 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:29:48.339323 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:29:48.339323 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:29:48.339323 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:29:48.339323 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:29:48.339323 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:29:48.339323 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:29:48.339323 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:29:48.339323 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:29:48.339323 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:29:48.340235 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:29:48.340235 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:29:48.340235 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:29:48.340235 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:29:48.340235 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:29:48.340235 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:29:48.340235 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:29:48.340235 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:29:48.340235 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:29:48.340235 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:29:48.340235 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:29:48.340235 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:29:48.340235 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:29:48.340235 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:29:48.340235 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:29:48.340235 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:29:48.341033 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:29:48.341033 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:29:48.341033 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:29:48.341033 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:29:48.341033 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:29:48.341033 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:29:48.341033 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:29:48.341033 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:29:48.341033 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:29:48.341033 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:29:48.341033 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 17:29:48.341033 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:29:48.341033 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 78/152] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Jun 16 17:29:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 386 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:29:48.346850 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69644da7-7823-4905-a752-d9a98fcea778 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:48.488168 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69644da7-7823-4905-a752-d9a98fcea778 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:48.507947 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:48.535596 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b78532-0536-4865-ac60-095fb1c6c5d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:48.536652 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b78532-0536-4865-ac60-095fb1c6c5d4 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:48.537117 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b78532-0536-4865-ac60-095fb1c6c5d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:48.537587 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b78532-0536-4865-ac60-095fb1c6c5d4 None admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:48.538570 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b78532-0536-4865-ac60-095fb1c6c5d4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:29:48.539108 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b78532-0536-4865-ac60-095fb1c6c5d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:48.540785 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b78532-0536-4865-ac60-095fb1c6c5d4 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:48.551079 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 75/153] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jun 16 17:29:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:29:48.557014 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84df9786-21e6-4649-a16a-d52778306f6c None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:48.671987 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:48.677078 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84df9786-21e6-4649-a16a-d52778306f6c None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:48.717031 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f23538-5f70-4827-a04b-89a80231b903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:48.717918 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f23538-5f70-4827-a04b-89a80231b903 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:48.718204 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f23538-5f70-4827-a04b-89a80231b903 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:48.718488 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f23538-5f70-4827-a04b-89a80231b903 None admin] PATH_INFO: `/v3/users/f72678162d55418190ac5579cb1eb41f` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:48.720543 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f23538-5f70-4827-a04b-89a80231b903 None admin] RBAC: Authorizing `identity:get_user(user_id=f72678162d55418190ac5579cb1eb41f)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:48.722048 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f23538-5f70-4827-a04b-89a80231b903 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:48.723759 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 79/154] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jun 16 17:29:48 2022] GET /identity/v3/users/f72678162d55418190ac5579cb1eb41f => generated 275 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:29:48.729367 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b78532-0536-4865-ac60-095fb1c6c5d4 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:48.871683 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b78532-0536-4865-ac60-095fb1c6c5d4 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:48.917606 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4165cd-1e75-40be-9f0f-2be86608179d None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:48.917917 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4165cd-1e75-40be-9f0f-2be86608179d None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:48.918172 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4165cd-1e75-40be-9f0f-2be86608179d None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:48.919566 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 80/155] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:29:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:29:48.927963 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9927c0d-8e13-4437-a77d-78a5a5d3cc8a None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:48.928248 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9927c0d-8e13-4437-a77d-78a5a5d3cc8a None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:48.928568 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9927c0d-8e13-4437-a77d-78a5a5d3cc8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:48.937366 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88139c00-aca7-474d-83c2-8a6a114f16b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:48.938809 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88139c00-aca7-474d-83c2-8a6a114f16b0 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:48.939131 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88139c00-aca7-474d-83c2-8a6a114f16b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:48.939409 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88139c00-aca7-474d-83c2-8a6a114f16b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:48.949733 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-a9927c0d-8e13-4437-a77d-78a5a5d3cc8a None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:29:48.950924 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-a9927c0d-8e13-4437-a77d-78a5a5d3cc8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:29:48.998295 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:49.038628 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88139c00-aca7-474d-83c2-8a6a114f16b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:49.040614 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88139c00-aca7-474d-83c2-8a6a114f16b0 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:49.045096 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 81/156] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:29:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:29:49.078942 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9927c0d-8e13-4437-a77d-78a5a5d3cc8a None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:49.178435 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 76/157] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jun 16 17:29:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:29:49.190492 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88139c00-aca7-474d-83c2-8a6a114f16b0 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:49.233480 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:49.242834 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9927c0d-8e13-4437-a77d-78a5a5d3cc8a None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:49.287983 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65802005-b641-4e5c-81b5-2b16bd3c6767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:49.289209 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65802005-b641-4e5c-81b5-2b16bd3c6767 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:49.289693 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65802005-b641-4e5c-81b5-2b16bd3c6767 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:49.290184 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65802005-b641-4e5c-81b5-2b16bd3c6767 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:49.303680 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65802005-b641-4e5c-81b5-2b16bd3c6767 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:29:49.311774 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-65802005-b641-4e5c-81b5-2b16bd3c6767 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 17:29:49.311774 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:29:49.311774 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:29:49.311774 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:29:49.311774 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:29:49.311774 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:29:49.311774 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:29:49.311774 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:29:49.311774 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:29:49.311774 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:29:49.311774 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:29:49.311774 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:29:49.311774 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:29:49.311774 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:29:49.311774 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:29:49.311774 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:29:49.313385 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:29:49.313385 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:29:49.313385 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:29:49.313385 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:29:49.313385 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:29:49.313385 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:29:49.313385 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:29:49.313385 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:29:49.313385 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:29:49.313385 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:29:49.313385 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:29:49.313385 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:29:49.313385 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:29:49.313385 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:29:49.313385 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:29:49.313385 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:29:49.314811 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:29:49.314811 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:29:49.314811 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:29:49.314811 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:29:49.314811 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:29:49.314811 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:29:49.314811 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:29:49.314811 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:29:49.314811 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:29:49.314811 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:29:49.314811 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 17:29:49.314811 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:29:49.317289 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 82/158] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jun 16 17:29:49 2022] GET /identity/v3/roles/reader => generated 84 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:29:49.318816 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172dcd01-9deb-4923-9729-85042c34b34d None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:49.319400 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172dcd01-9deb-4923-9729-85042c34b34d None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:49.319867 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172dcd01-9deb-4923-9729-85042c34b34d None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:49.324384 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:49.324799 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 83/159] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:29:49 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:29:49.326848 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172dcd01-9deb-4923-9729-85042c34b34d None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:49.350338 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88139c00-aca7-474d-83c2-8a6a114f16b0 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:49.393944 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7a2f5c-936b-41dd-b33e-f6f352e90842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:49.395195 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7a2f5c-936b-41dd-b33e-f6f352e90842 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:49.395676 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7a2f5c-936b-41dd-b33e-f6f352e90842 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:49.396166 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7a2f5c-936b-41dd-b33e-f6f352e90842 None admin] PATH_INFO: `/v3/projects/3be66087f3954ee4bac1dc53ea4d59ed` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:49.399485 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7a2f5c-936b-41dd-b33e-f6f352e90842 None admin] RBAC: Authorizing `identity:get_project(project_id=3be66087f3954ee4bac1dc53ea4d59ed)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:49.401238 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7a2f5c-936b-41dd-b33e-f6f352e90842 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:49.403190 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 77/160] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jun 16 17:29:49 2022] GET /identity/v3/projects/3be66087f3954ee4bac1dc53ea4d59ed => generated 298 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:29:49.404634 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d341f8-9240-447c-9a80-2feb9d817afd None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:49.405147 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d341f8-9240-447c-9a80-2feb9d817afd None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:49.405632 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d341f8-9240-447c-9a80-2feb9d817afd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:49.429385 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-b5d341f8-9240-447c-9a80-2feb9d817afd None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:29:49.430891 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-b5d341f8-9240-447c-9a80-2feb9d817afd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:29:49.468856 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172dcd01-9deb-4923-9729-85042c34b34d None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:49.505887 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240abb3c-28eb-4d76-8e73-d386296b3c09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:49.506566 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240abb3c-28eb-4d76-8e73-d386296b3c09 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:49.506844 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240abb3c-28eb-4d76-8e73-d386296b3c09 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:49.507150 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240abb3c-28eb-4d76-8e73-d386296b3c09 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:49.507857 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240abb3c-28eb-4d76-8e73-d386296b3c09 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:29:49.508158 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240abb3c-28eb-4d76-8e73-d386296b3c09 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:49.509591 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240abb3c-28eb-4d76-8e73-d386296b3c09 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:49.511780 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 78/161] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:29:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:29:49.513197 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d341f8-9240-447c-9a80-2feb9d817afd None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:49.520867 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 84/162] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Jun 16 17:29:49 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:29:49.522284 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9813a0-2c39-4b3b-b5da-0c1b4cf5ab13 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:49.522661 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9813a0-2c39-4b3b-b5da-0c1b4cf5ab13 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:49.522945 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9813a0-2c39-4b3b-b5da-0c1b4cf5ab13 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:49.524265 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 85/163] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:29:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:29:49.527793 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9813a0-2c39-4b3b-b5da-0c1b4cf5ab13 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:49.557557 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:49.605863 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:49.638040 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d341f8-9240-447c-9a80-2feb9d817afd None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:49.684227 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc346d86-5b18-49f5-8cea-12456d3211f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:49.684891 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc346d86-5b18-49f5-8cea-12456d3211f5 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:49.685180 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc346d86-5b18-49f5-8cea-12456d3211f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:49.685454 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc346d86-5b18-49f5-8cea-12456d3211f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:49.686303 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc346d86-5b18-49f5-8cea-12456d3211f5 None admin] RBAC: Adding query filter params (role.id=243fd0a11d94477986deb1dcfb3f31dd, scope.project.id=3be66087f3954ee4bac1dc53ea4d59ed, user.id=f72678162d55418190ac5579cb1eb41f) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:29:49.686617 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc346d86-5b18-49f5-8cea-12456d3211f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:49.688277 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc346d86-5b18-49f5-8cea-12456d3211f5 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:49.692419 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9813a0-2c39-4b3b-b5da-0c1b4cf5ab13 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:49.697845 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 79/164] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jun 16 17:29:49 2022] GET /identity/v3/role_assignments?user.id=f72678162d55418190ac5579cb1eb41f&scope.project.id=3be66087f3954ee4bac1dc53ea4d59ed&role.id=243fd0a11d94477986deb1dcfb3f31dd => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:29:49.699473 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc5650e-ce65-41ed-a0ba-0a18ecdcc4de None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:49.699917 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc5650e-ce65-41ed-a0ba-0a18ecdcc4de None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:49.700339 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc5650e-ce65-41ed-a0ba-0a18ecdcc4de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:49.719170 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-1fc5650e-ce65-41ed-a0ba-0a18ecdcc4de None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:29:49.720721 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-1fc5650e-ce65-41ed-a0ba-0a18ecdcc4de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:29:49.760404 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676f0172-8255-4960-8231-67dc218a9624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:49.763108 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676f0172-8255-4960-8231-67dc218a9624 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:49.765187 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676f0172-8255-4960-8231-67dc218a9624 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:49.770635 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676f0172-8255-4960-8231-67dc218a9624 None admin] PATH_INFO: `/v3/users/ebd6465c9b4f4f4aaed32bb07032d7f0` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:49.774845 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676f0172-8255-4960-8231-67dc218a9624 None admin] RBAC: Authorizing `identity:get_user(user_id=ebd6465c9b4f4f4aaed32bb07032d7f0)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:49.776591 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676f0172-8255-4960-8231-67dc218a9624 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:49.778578 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 86/165] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jun 16 17:29:49 2022] GET /identity/v3/users/ebd6465c9b4f4f4aaed32bb07032d7f0 => generated 289 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:29:49.779539 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676f0172-8255-4960-8231-67dc218a9624 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:49.840397 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 80/166] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:29:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:29:49.840397 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc5650e-ce65-41ed-a0ba-0a18ecdcc4de None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:49.958766 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:49.981791 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676f0172-8255-4960-8231-67dc218a9624 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:49.991823 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:50.065904 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc5650e-ce65-41ed-a0ba-0a18ecdcc4de None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:50.071778 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dc9933-b809-46f5-a52f-aa8f571ec226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:50.072648 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dc9933-b809-46f5-a52f-aa8f571ec226 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:50.072934 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dc9933-b809-46f5-a52f-aa8f571ec226 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:50.073212 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dc9933-b809-46f5-a52f-aa8f571ec226 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:50.088108 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dc9933-b809-46f5-a52f-aa8f571ec226 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:29:50.098237 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-00dc9933-b809-46f5-a52f-aa8f571ec226 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:29:50.098237 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:29:50.098237 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:29:50.098237 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:29:50.098237 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:29:50.098237 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:29:50.098237 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:29:50.098237 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:29:50.098237 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:29:50.098237 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:29:50.098237 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:29:50.098237 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:29:50.098237 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:29:50.098237 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:29:50.098237 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:29:50.098237 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:29:50.099236 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:29:50.099236 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:29:50.099236 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:29:50.099236 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:29:50.099236 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:29:50.099236 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:29:50.099236 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:29:50.099236 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:29:50.099236 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:29:50.099236 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:29:50.099236 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:29:50.099236 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:29:50.099236 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:29:50.099236 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:29:50.099236 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:29:50.099236 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:29:50.100156 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:29:50.100156 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:29:50.100156 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:29:50.100156 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:29:50.100156 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:29:50.100156 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:29:50.100156 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:29:50.100156 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:29:50.100156 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:29:50.100156 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:29:50.100156 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:29:50.100156 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:29:50.100156 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 87/167] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Jun 16 17:29:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:29:50.100859 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dc9933-b809-46f5-a52f-aa8f571ec226 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:50.148809 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8542d183-04b7-4ec1-9c65-8633c7ab284a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:50.149810 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8542d183-04b7-4ec1-9c65-8633c7ab284a None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:50.150105 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8542d183-04b7-4ec1-9c65-8633c7ab284a None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:50.150380 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8542d183-04b7-4ec1-9c65-8633c7ab284a None admin] PATH_INFO: `/v3/projects/3be66087f3954ee4bac1dc53ea4d59ed` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:50.151665 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8542d183-04b7-4ec1-9c65-8633c7ab284a None admin] RBAC: Authorizing `identity:get_project(project_id=3be66087f3954ee4bac1dc53ea4d59ed)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:50.153176 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8542d183-04b7-4ec1-9c65-8633c7ab284a None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:50.155202 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 81/168] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jun 16 17:29:49 2022] GET /identity/v3/projects/3be66087f3954ee4bac1dc53ea4d59ed => generated 298 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:29:50.155819 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8542d183-04b7-4ec1-9c65-8633c7ab284a None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:50.236434 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dc9933-b809-46f5-a52f-aa8f571ec226 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:50.277366 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c5e059-8d6a-4977-b683-943d49af771c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:50.278342 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c5e059-8d6a-4977-b683-943d49af771c None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:50.278627 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c5e059-8d6a-4977-b683-943d49af771c None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:50.279047 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c5e059-8d6a-4977-b683-943d49af771c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:50.290808 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c5e059-8d6a-4977-b683-943d49af771c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:29:50.299211 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-99c5e059-8d6a-4977-b683-943d49af771c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:29:50.299211 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:29:50.299211 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:29:50.299211 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:29:50.299211 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:29:50.299211 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:29:50.299211 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:29:50.299211 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:29:50.299211 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:29:50.299211 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:29:50.299211 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:29:50.299211 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:29:50.299211 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:29:50.299211 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:29:50.299211 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:29:50.299211 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:29:50.300109 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:29:50.300109 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:29:50.300109 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:29:50.300109 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:29:50.300109 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:29:50.300109 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:29:50.300109 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:29:50.300109 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:29:50.300109 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:29:50.300109 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:29:50.300109 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:29:50.300109 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:29:50.300109 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:29:50.300109 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:29:50.300109 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:29:50.300109 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:29:50.300946 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:29:50.300946 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:29:50.300946 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:29:50.300946 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:29:50.300946 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:29:50.300946 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:29:50.300946 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:29:50.300946 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:29:50.300946 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:29:50.300946 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:29:50.300946 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:29:50.300946 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:29:50.301542 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 88/169] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jun 16 17:29:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:29:50.302425 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c5e059-8d6a-4977-b683-943d49af771c None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:50.312687 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:50.426494 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c5e059-8d6a-4977-b683-943d49af771c None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:50.458575 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8542d183-04b7-4ec1-9c65-8633c7ab284a None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:50.466827 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15f979d-d8de-430c-883e-e78cd9a66b6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:50.467732 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15f979d-d8de-430c-883e-e78cd9a66b6c None admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:50.468016 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15f979d-d8de-430c-883e-e78cd9a66b6c None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:50.468293 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15f979d-d8de-430c-883e-e78cd9a66b6c None admin] PATH_INFO: `/v3/projects/3be66087f3954ee4bac1dc53ea4d59ed/users/ebd6465c9b4f4f4aaed32bb07032d7f0/roles/9ed5382e55d649faa9195c4a9eb6fa57` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:50.471269 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15f979d-d8de-430c-883e-e78cd9a66b6c None admin] RBAC: Authorizing `identity:create_grant(project_id=3be66087f3954ee4bac1dc53ea4d59ed, user_id=ebd6465c9b4f4f4aaed32bb07032d7f0, role_id=9ed5382e55d649faa9195c4a9eb6fa57)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:50.473281 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15f979d-d8de-430c-883e-e78cd9a66b6c None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:50.489072 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 89/170] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Jun 16 17:29:50 2022] PUT /identity/v3/projects/3be66087f3954ee4bac1dc53ea4d59ed/users/ebd6465c9b4f4f4aaed32bb07032d7f0/roles/9ed5382e55d649faa9195c4a9eb6fa57 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:29:50.489885 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15f979d-d8de-430c-883e-e78cd9a66b6c None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:50.545238 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743c7277-2c6e-45c5-8856-f815153ed27f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:50.545947 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743c7277-2c6e-45c5-8856-f815153ed27f None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:50.546231 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743c7277-2c6e-45c5-8856-f815153ed27f None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:50.546506 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743c7277-2c6e-45c5-8856-f815153ed27f None admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:50.547469 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743c7277-2c6e-45c5-8856-f815153ed27f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:29:50.547663 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743c7277-2c6e-45c5-8856-f815153ed27f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:50.549197 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743c7277-2c6e-45c5-8856-f815153ed27f None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:50.551766 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:50.574681 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 82/171] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jun 16 17:29:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 419 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:29:50.581239 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743c7277-2c6e-45c5-8856-f815153ed27f None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:50.589680 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:50.637033 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15f979d-d8de-430c-883e-e78cd9a66b6c None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:50.679492 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b969e5d-f34c-49b9-a583-059809068a0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:50.680182 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b969e5d-f34c-49b9-a583-059809068a0f None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:50.680493 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b969e5d-f34c-49b9-a583-059809068a0f None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:50.680890 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b969e5d-f34c-49b9-a583-059809068a0f None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:50.681666 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b969e5d-f34c-49b9-a583-059809068a0f None admin] RBAC: Adding query filter params (name=member) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:29:50.682221 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b969e5d-f34c-49b9-a583-059809068a0f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:50.683631 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b969e5d-f34c-49b9-a583-059809068a0f None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:50.694477 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 90/172] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Jun 16 17:29:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:29:50.699831 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b969e5d-f34c-49b9-a583-059809068a0f None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:50.841059 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b969e5d-f34c-49b9-a583-059809068a0f None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:50.873860 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:50.889990 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98de15d-c42b-4141-b251-384229a77bc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:50.891232 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98de15d-c42b-4141-b251-384229a77bc3 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:50.892055 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98de15d-c42b-4141-b251-384229a77bc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:50.892335 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98de15d-c42b-4141-b251-384229a77bc3 None admin] PATH_INFO: `/v3/users/f72678162d55418190ac5579cb1eb41f` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:50.893954 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98de15d-c42b-4141-b251-384229a77bc3 None admin] RBAC: Authorizing `identity:get_user(user_id=f72678162d55418190ac5579cb1eb41f)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:50.894766 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743c7277-2c6e-45c5-8856-f815153ed27f None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:50.895644 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98de15d-c42b-4141-b251-384229a77bc3 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:50.897979 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 91/173] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jun 16 17:29:50 2022] GET /identity/v3/users/f72678162d55418190ac5579cb1eb41f => generated 275 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:29:50.906210 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98de15d-c42b-4141-b251-384229a77bc3 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:51.017735 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734b0f14-98af-40f6-8d2e-57aeb383a66e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:51.018630 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734b0f14-98af-40f6-8d2e-57aeb383a66e None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:51.018913 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734b0f14-98af-40f6-8d2e-57aeb383a66e None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:51.019187 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734b0f14-98af-40f6-8d2e-57aeb383a66e None admin] PATH_INFO: `/v3/users/12a220316b144048ad50f0affd87fa43` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:51.025162 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734b0f14-98af-40f6-8d2e-57aeb383a66e None admin] RBAC: Authorizing `identity:get_user(user_id=12a220316b144048ad50f0affd87fa43)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:51.026641 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734b0f14-98af-40f6-8d2e-57aeb383a66e None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:51.028859 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 83/174] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jun 16 17:29:50 2022] GET /identity/v3/users/12a220316b144048ad50f0affd87fa43 => generated 247 bytes in 448 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:29:51.034928 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734b0f14-98af-40f6-8d2e-57aeb383a66e None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:51.040006 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98de15d-c42b-4141-b251-384229a77bc3 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:51.084405 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0310ab-c3ae-4025-8555-e784e2c35ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:51.085248 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0310ab-c3ae-4025-8555-e784e2c35ffc None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:51.085358 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0310ab-c3ae-4025-8555-e784e2c35ffc None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:51.085676 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0310ab-c3ae-4025-8555-e784e2c35ffc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:51.167499 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:51.178416 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0310ab-c3ae-4025-8555-e784e2c35ffc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:51.180046 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0310ab-c3ae-4025-8555-e784e2c35ffc None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:51.243140 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:51.307969 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 92/175] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jun 16 17:29:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:29:51.314223 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0310ab-c3ae-4025-8555-e784e2c35ffc None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:51.327168 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734b0f14-98af-40f6-8d2e-57aeb383a66e None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:51.409270 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3107ef6b-758c-42f5-9c72-cd4ec27621ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:51.410303 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3107ef6b-758c-42f5-9c72-cd4ec27621ca None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:51.410790 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3107ef6b-758c-42f5-9c72-cd4ec27621ca None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:51.411265 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3107ef6b-758c-42f5-9c72-cd4ec27621ca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:51.419476 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:51.427064 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0310ab-c3ae-4025-8555-e784e2c35ffc None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:51.463386 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12598586-4fa8-49c0-8b64-0c1d351d759c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:51.464516 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12598586-4fa8-49c0-8b64-0c1d351d759c None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:51.465023 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12598586-4fa8-49c0-8b64-0c1d351d759c None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:51.465541 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12598586-4fa8-49c0-8b64-0c1d351d759c None admin] PATH_INFO: `/v3/projects/afad0aa3b7c040bda29e672a36907ecb` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:51.466973 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12598586-4fa8-49c0-8b64-0c1d351d759c None admin] RBAC: Authorizing `identity:get_project(project_id=afad0aa3b7c040bda29e672a36907ecb)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:51.468656 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12598586-4fa8-49c0-8b64-0c1d351d759c None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:51.471354 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 93/176] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jun 16 17:29:51 2022] GET /identity/v3/projects/afad0aa3b7c040bda29e672a36907ecb => generated 312 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:29:51.480585 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12598586-4fa8-49c0-8b64-0c1d351d759c None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:51.585320 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3107ef6b-758c-42f5-9c72-cd4ec27621ca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:51.586958 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3107ef6b-758c-42f5-9c72-cd4ec27621ca None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:51.604116 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12598586-4fa8-49c0-8b64-0c1d351d759c None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:51.652995 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44c088e-83e5-4ca6-b1e7-e7f06809929b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:51.654550 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44c088e-83e5-4ca6-b1e7-e7f06809929b None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:51.655336 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44c088e-83e5-4ca6-b1e7-e7f06809929b None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:51.656086 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44c088e-83e5-4ca6-b1e7-e7f06809929b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:51.659065 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44c088e-83e5-4ca6-b1e7-e7f06809929b None admin] RBAC: Adding query filter params (role.id=0be2075946f54c8485f1c275f746478a, scope.project.id=afad0aa3b7c040bda29e672a36907ecb, user.id=f72678162d55418190ac5579cb1eb41f) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:29:51.659955 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44c088e-83e5-4ca6-b1e7-e7f06809929b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:51.663140 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44c088e-83e5-4ca6-b1e7-e7f06809929b None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:51.676867 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 94/177] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jun 16 17:29:51 2022] GET /identity/v3/role_assignments?user.id=f72678162d55418190ac5579cb1eb41f&scope.project.id=afad0aa3b7c040bda29e672a36907ecb&role.id=0be2075946f54c8485f1c275f746478a => generated 608 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:29:51.745721 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:51.807974 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282160d8-1997-442e-82de-7fdbd940c8e0 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:51.808617 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282160d8-1997-442e-82de-7fdbd940c8e0 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:51.808902 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282160d8-1997-442e-82de-7fdbd940c8e0 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:51.810304 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 95/178] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:29:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:29:51.813332 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 84/179] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jun 16 17:29:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 779 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 17:29:51.821828 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282160d8-1997-442e-82de-7fdbd940c8e0 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:51.826276 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21207db-d808-4f8e-9a0e-83997aac292f None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:51.826632 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21207db-d808-4f8e-9a0e-83997aac292f None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:51.826868 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21207db-d808-4f8e-9a0e-83997aac292f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:51.846501 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-b21207db-d808-4f8e-9a0e-83997aac292f None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:29:51.847851 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-b21207db-d808-4f8e-9a0e-83997aac292f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:29:51.914121 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:51.983075 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 85/180] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:29:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 17:29:52.030202 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21207db-d808-4f8e-9a0e-83997aac292f None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:52.089043 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282160d8-1997-442e-82de-7fdbd940c8e0 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:52.166802 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b11f512-dd9f-43d2-a1ac-e6100258ec79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:52.168085 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b11f512-dd9f-43d2-a1ac-e6100258ec79 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:52.168583 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b11f512-dd9f-43d2-a1ac-e6100258ec79 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:52.169072 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b11f512-dd9f-43d2-a1ac-e6100258ec79 None admin] PATH_INFO: `/v3/projects/3be66087f3954ee4bac1dc53ea4d59ed` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:52.172806 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b11f512-dd9f-43d2-a1ac-e6100258ec79 None admin] RBAC: Authorizing `identity:get_project(project_id=3be66087f3954ee4bac1dc53ea4d59ed)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:52.174522 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b11f512-dd9f-43d2-a1ac-e6100258ec79 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:52.176508 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 96/181] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jun 16 17:29:51 2022] GET /identity/v3/projects/3be66087f3954ee4bac1dc53ea4d59ed => generated 298 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:29:52.185261 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b11f512-dd9f-43d2-a1ac-e6100258ec79 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:52.198478 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:52.207355 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21207db-d808-4f8e-9a0e-83997aac292f None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:52.289135 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203fe683-936e-499d-afd9-1b95ba31a7c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:52.289905 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203fe683-936e-499d-afd9-1b95ba31a7c3 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:52.290195 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203fe683-936e-499d-afd9-1b95ba31a7c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:52.290500 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203fe683-936e-499d-afd9-1b95ba31a7c3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:52.314843 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203fe683-936e-499d-afd9-1b95ba31a7c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:29:52.322979 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-203fe683-936e-499d-afd9-1b95ba31a7c3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:29:52.322979 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:29:52.322979 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:29:52.322979 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:29:52.322979 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:29:52.322979 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:29:52.322979 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:29:52.322979 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:29:52.322979 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:29:52.322979 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:29:52.322979 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:29:52.322979 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:29:52.322979 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:29:52.322979 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:29:52.322979 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:29:52.322979 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:29:52.323985 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:29:52.323985 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:29:52.323985 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:29:52.323985 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:29:52.323985 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:29:52.323985 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:29:52.323985 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:29:52.323985 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:29:52.323985 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:29:52.323985 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:29:52.323985 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:29:52.323985 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:29:52.323985 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:29:52.323985 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:29:52.323985 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:29:52.323985 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:29:52.324785 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:29:52.324785 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:29:52.324785 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:29:52.324785 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:29:52.324785 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:29:52.324785 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:29:52.324785 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:29:52.324785 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:29:52.324785 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:29:52.324785 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:29:52.324785 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:29:52.324785 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:29:52.335981 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 86/182] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Jun 16 17:29:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:29:52.336966 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203fe683-936e-499d-afd9-1b95ba31a7c3 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:52.392480 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:52.402394 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b11f512-dd9f-43d2-a1ac-e6100258ec79 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:52.488174 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb36341d-a740-4519-a7de-ed7300b72c10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:52.489215 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb36341d-a740-4519-a7de-ed7300b72c10 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:52.491878 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb36341d-a740-4519-a7de-ed7300b72c10 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:52.493519 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb36341d-a740-4519-a7de-ed7300b72c10 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:52.495217 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb36341d-a740-4519-a7de-ed7300b72c10 None admin] RBAC: Adding query filter params (role.id=3ca2dcbbd29d4a31a3ab78f2299bc239, scope.project.id=3be66087f3954ee4bac1dc53ea4d59ed, user.id=12a220316b144048ad50f0affd87fa43) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:29:52.495919 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb36341d-a740-4519-a7de-ed7300b72c10 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:52.499116 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb36341d-a740-4519-a7de-ed7300b72c10 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:52.513309 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 97/183] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jun 16 17:29:52 2022] GET /identity/v3/role_assignments?user.id=12a220316b144048ad50f0affd87fa43&scope.project.id=3be66087f3954ee4bac1dc53ea4d59ed&role.id=3ca2dcbbd29d4a31a3ab78f2299bc239 => generated 608 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:29:52.526956 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203fe683-936e-499d-afd9-1b95ba31a7c3 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:52.564730 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddf90c5-c673-4e21-9e33-698a7d82c839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:52.565755 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddf90c5-c673-4e21-9e33-698a7d82c839 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:52.566232 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddf90c5-c673-4e21-9e33-698a7d82c839 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:52.566719 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddf90c5-c673-4e21-9e33-698a7d82c839 None admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:52.567687 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddf90c5-c673-4e21-9e33-698a7d82c839 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:29:52.568195 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddf90c5-c673-4e21-9e33-698a7d82c839 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:52.569865 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddf90c5-c673-4e21-9e33-698a7d82c839 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:52.580148 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 87/184] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jun 16 17:29:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:29:52.586018 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb36341d-a740-4519-a7de-ed7300b72c10 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:52.661642 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5702ba6f-39f9-4c0b-a7c8-b172715f1f02 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:52.661996 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5702ba6f-39f9-4c0b-a7c8-b172715f1f02 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:52.662218 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5702ba6f-39f9-4c0b-a7c8-b172715f1f02 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:52.663664 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 88/185] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:29:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:29:52.678812 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f46096-526b-401b-8b0e-a375690ca07e None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:52.679121 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f46096-526b-401b-8b0e-a375690ca07e None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:52.679380 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f46096-526b-401b-8b0e-a375690ca07e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:52.709549 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb36341d-a740-4519-a7de-ed7300b72c10 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:52.725610 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-87f46096-526b-401b-8b0e-a375690ca07e None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:29:52.727211 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-87f46096-526b-401b-8b0e-a375690ca07e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:29:52.749779 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5be4317-6d4f-43a4-a1dd-0fa412d22003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:52.751689 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5be4317-6d4f-43a4-a1dd-0fa412d22003 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:52.751983 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:52.752243 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5be4317-6d4f-43a4-a1dd-0fa412d22003 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:52.752558 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5be4317-6d4f-43a4-a1dd-0fa412d22003 None admin] PATH_INFO: `/v3/users/bc760a5800b649989e800249b203636f` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:52.769364 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5be4317-6d4f-43a4-a1dd-0fa412d22003 None admin] RBAC: Authorizing `identity:get_user(user_id=bc760a5800b649989e800249b203636f)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:52.771110 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5be4317-6d4f-43a4-a1dd-0fa412d22003 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:52.777075 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 98/186] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jun 16 17:29:52 2022] GET /identity/v3/users/bc760a5800b649989e800249b203636f => generated 283 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:29:52.780686 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5be4317-6d4f-43a4-a1dd-0fa412d22003 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:52.807444 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:52.860734 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 89/187] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:29:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 17:29:52.894476 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f46096-526b-401b-8b0e-a375690ca07e None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:52.924002 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5be4317-6d4f-43a4-a1dd-0fa412d22003 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:52.960693 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dbe7ca-5deb-4b2b-8745-3c3bd5c1d424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:52.961366 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dbe7ca-5deb-4b2b-8745-3c3bd5c1d424 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:52.961653 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dbe7ca-5deb-4b2b-8745-3c3bd5c1d424 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:52.961937 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dbe7ca-5deb-4b2b-8745-3c3bd5c1d424 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:53.019561 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f46096-526b-401b-8b0e-a375690ca07e None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:53.044841 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dbe7ca-5deb-4b2b-8745-3c3bd5c1d424 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:53.046445 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dbe7ca-5deb-4b2b-8745-3c3bd5c1d424 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:53.058214 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecf5900-cb8e-4261-a594-306ca33c1052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:53.058824 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecf5900-cb8e-4261-a594-306ca33c1052 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:53.059095 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecf5900-cb8e-4261-a594-306ca33c1052 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:53.059368 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecf5900-cb8e-4261-a594-306ca33c1052 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:53.067629 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:53.076225 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecf5900-cb8e-4261-a594-306ca33c1052 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:29:53.088573 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-1ecf5900-cb8e-4261-a594-306ca33c1052 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 17:29:53.088573 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:29:53.088573 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:29:53.088573 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:29:53.088573 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:29:53.088573 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:29:53.088573 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:29:53.088573 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:29:53.088573 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:29:53.088573 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:29:53.088573 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:29:53.088573 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:29:53.088573 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:29:53.088573 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:29:53.088573 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:29:53.088573 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:29:53.089683 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:29:53.089683 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:29:53.089683 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:29:53.089683 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:29:53.089683 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:29:53.089683 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:29:53.089683 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:29:53.089683 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:29:53.089683 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:29:53.089683 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:29:53.089683 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:29:53.089683 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:29:53.089683 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:29:53.089683 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:29:53.089683 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:29:53.089683 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:29:53.090641 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:29:53.090641 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:29:53.090641 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:29:53.090641 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:29:53.090641 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:29:53.090641 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:29:53.090641 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:29:53.090641 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:29:53.090641 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:29:53.090641 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:29:53.090641 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 17:29:53.090641 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:29:53.093199 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 90/188] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Jun 16 17:29:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:29:53.099142 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b45acc-95fb-4958-a085-28719826fe99 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:53.099142 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b45acc-95fb-4958-a085-28719826fe99 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:53.099341 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b45acc-95fb-4958-a085-28719826fe99 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:53.104078 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 91/189] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:29:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:29:53.104078 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b45acc-95fb-4958-a085-28719826fe99 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:53.114462 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:53.186620 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 99/190] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jun 16 17:29:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:29:53.187711 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a168fda-5004-4180-b425-8b5b5fe38316 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:53.188204 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a168fda-5004-4180-b425-8b5b5fe38316 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:53.188607 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a168fda-5004-4180-b425-8b5b5fe38316 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:53.207145 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-2a168fda-5004-4180-b425-8b5b5fe38316 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:29:53.209272 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-2a168fda-5004-4180-b425-8b5b5fe38316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:29:53.237977 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b45acc-95fb-4958-a085-28719826fe99 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:53.280917 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aed636b-7bea-4934-9706-ded4bebc3bd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:53.282718 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aed636b-7bea-4934-9706-ded4bebc3bd4 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:53.283316 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aed636b-7bea-4934-9706-ded4bebc3bd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:53.283938 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aed636b-7bea-4934-9706-ded4bebc3bd4 None admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:53.285228 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aed636b-7bea-4934-9706-ded4bebc3bd4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:29:53.286004 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aed636b-7bea-4934-9706-ded4bebc3bd4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:53.287810 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aed636b-7bea-4934-9706-ded4bebc3bd4 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:53.289609 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 100/191] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:29:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:29:53.291045 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a168fda-5004-4180-b425-8b5b5fe38316 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:53.313865 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 92/192] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jun 16 17:29:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:29:53.323363 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aed636b-7bea-4934-9706-ded4bebc3bd4 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:53.371099 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:53.407169 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:53.431642 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a168fda-5004-4180-b425-8b5b5fe38316 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:53.450349 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aed636b-7bea-4934-9706-ded4bebc3bd4 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:53.472252 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d835603d-b8f8-4a70-9230-98606238632e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:53.473170 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d835603d-b8f8-4a70-9230-98606238632e None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:53.473452 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d835603d-b8f8-4a70-9230-98606238632e None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:53.473729 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d835603d-b8f8-4a70-9230-98606238632e None admin] PATH_INFO: `/v3/projects/d820cdcb6c0741e380190e6fe127e992` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:53.475078 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d835603d-b8f8-4a70-9230-98606238632e None admin] RBAC: Authorizing `identity:get_project(project_id=d820cdcb6c0741e380190e6fe127e992)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:53.476573 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d835603d-b8f8-4a70-9230-98606238632e None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:53.483753 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 101/193] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jun 16 17:29:53 2022] GET /identity/v3/projects/d820cdcb6c0741e380190e6fe127e992 => generated 302 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:29:53.485487 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d835603d-b8f8-4a70-9230-98606238632e None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:53.488921 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b3ab27-82ae-4131-a255-3c3d2a77c997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:53.490715 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b3ab27-82ae-4131-a255-3c3d2a77c997 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:53.491001 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b3ab27-82ae-4131-a255-3c3d2a77c997 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:53.491278 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b3ab27-82ae-4131-a255-3c3d2a77c997 None admin] PATH_INFO: `/v3/users/bc760a5800b649989e800249b203636f` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:53.492536 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b3ab27-82ae-4131-a255-3c3d2a77c997 None admin] RBAC: Authorizing `identity:get_user(user_id=bc760a5800b649989e800249b203636f)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:53.493990 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b3ab27-82ae-4131-a255-3c3d2a77c997 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:53.501564 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 93/194] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jun 16 17:29:53 2022] GET /identity/v3/users/bc760a5800b649989e800249b203636f => generated 283 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:29:53.501564 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b3ab27-82ae-4131-a255-3c3d2a77c997 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:53.587108 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d835603d-b8f8-4a70-9230-98606238632e None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:53.600950 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b3ab27-82ae-4131-a255-3c3d2a77c997 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:53.603400 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:53.622120 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3563c06-dc5c-43c5-82ea-126d5fc12e4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:53.622835 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3563c06-dc5c-43c5-82ea-126d5fc12e4b None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:53.623116 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3563c06-dc5c-43c5-82ea-126d5fc12e4b None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:53.623390 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3563c06-dc5c-43c5-82ea-126d5fc12e4b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:53.624588 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3563c06-dc5c-43c5-82ea-126d5fc12e4b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:53.626363 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3563c06-dc5c-43c5-82ea-126d5fc12e4b None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:53.627996 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 102/195] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jun 16 17:29:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:29:53.632496 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3563c06-dc5c-43c5-82ea-126d5fc12e4b None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:53.635635 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8364fcf-1e5e-44b1-9044-3f145bda14bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:53.636316 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8364fcf-1e5e-44b1-9044-3f145bda14bf None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:53.636631 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8364fcf-1e5e-44b1-9044-3f145bda14bf None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:53.636932 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8364fcf-1e5e-44b1-9044-3f145bda14bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:53.637823 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8364fcf-1e5e-44b1-9044-3f145bda14bf None admin] RBAC: Adding query filter params (role.id=3ca2dcbbd29d4a31a3ab78f2299bc239, scope.project.id=d820cdcb6c0741e380190e6fe127e992, user.id=bc760a5800b649989e800249b203636f) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:29:53.638390 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8364fcf-1e5e-44b1-9044-3f145bda14bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:53.639831 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8364fcf-1e5e-44b1-9044-3f145bda14bf None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:53.643454 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:53.646986 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 94/196] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jun 16 17:29:53 2022] GET /identity/v3/role_assignments?user.id=bc760a5800b649989e800249b203636f&scope.project.id=d820cdcb6c0741e380190e6fe127e992&role.id=3ca2dcbbd29d4a31a3ab78f2299bc239 => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:29:53.648408 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cefb57-9e02-4244-84fa-681dc573b4a4 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:53.648721 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cefb57-9e02-4244-84fa-681dc573b4a4 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:53.649291 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cefb57-9e02-4244-84fa-681dc573b4a4 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:53.650891 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 95/197] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:29:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:29:53.651542 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cefb57-9e02-4244-84fa-681dc573b4a4 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:53.754430 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3563c06-dc5c-43c5-82ea-126d5fc12e4b None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:53.772328 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cefb57-9e02-4244-84fa-681dc573b4a4 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:53.792787 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29d4271-d5a7-4dd6-a0b7-5e3646be5149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:53.793455 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29d4271-d5a7-4dd6-a0b7-5e3646be5149 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:53.793745 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29d4271-d5a7-4dd6-a0b7-5e3646be5149 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:53.794030 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29d4271-d5a7-4dd6-a0b7-5e3646be5149 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:53.818784 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eff1b8-3f92-4c62-9c33-994bf43dd76e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:53.819520 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eff1b8-3f92-4c62-9c33-994bf43dd76e None admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:53.819841 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eff1b8-3f92-4c62-9c33-994bf43dd76e None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:53.820115 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eff1b8-3f92-4c62-9c33-994bf43dd76e None admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:53.820877 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9eff1b8-3f92-4c62-9c33-994bf43dd76e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:53.822336 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9eff1b8-3f92-4c62-9c33-994bf43dd76e None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:53.844711 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 96/198] 10.222.0.42 () {60 vars in 1132 bytes} [Thu Jun 16 17:29:53 2022] POST /identity/v3/users => generated 297 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:29:53.846092 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f801b94-c33b-4a49-81e4-9c4e86cad038 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:53.846464 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f801b94-c33b-4a49-81e4-9c4e86cad038 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:53.846756 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f801b94-c33b-4a49-81e4-9c4e86cad038 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:53.862358 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:53.867888 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-0f801b94-c33b-4a49-81e4-9c4e86cad038 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:29:53.869136 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-0f801b94-c33b-4a49-81e4-9c4e86cad038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:29:53.899758 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29d4271-d5a7-4dd6-a0b7-5e3646be5149 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:53.899758 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29d4271-d5a7-4dd6-a0b7-5e3646be5149 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:53.940794 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:53.961595 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 97/199] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:29:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:29:53.988597 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f801b94-c33b-4a49-81e4-9c4e86cad038 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:54.015771 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 103/200] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jun 16 17:29:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:29:54.022288 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29d4271-d5a7-4dd6-a0b7-5e3646be5149 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:54.121735 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f801b94-c33b-4a49-81e4-9c4e86cad038 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:54.133101 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:54.153620 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29d4271-d5a7-4dd6-a0b7-5e3646be5149 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:54.166558 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33da2bab-fc6c-4384-ae65-df0092779b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:54.167735 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33da2bab-fc6c-4384-ae65-df0092779b49 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:54.168603 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33da2bab-fc6c-4384-ae65-df0092779b49 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:54.169075 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33da2bab-fc6c-4384-ae65-df0092779b49 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:54.187641 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33da2bab-fc6c-4384-ae65-df0092779b49 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:29:54.196129 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-33da2bab-fc6c-4384-ae65-df0092779b49 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 17:29:54.196129 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:29:54.196129 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:29:54.196129 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:29:54.196129 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:29:54.196129 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:29:54.196129 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:29:54.196129 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:29:54.196129 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:29:54.196129 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:29:54.196129 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:29:54.196129 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:29:54.196129 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:29:54.196129 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:29:54.196129 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:29:54.196129 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:29:54.197148 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:29:54.197148 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:29:54.197148 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:29:54.197148 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:29:54.197148 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:29:54.197148 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:29:54.197148 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:29:54.197148 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:29:54.197148 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:29:54.197148 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:29:54.197148 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:29:54.197148 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:29:54.197148 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:29:54.197148 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:29:54.197148 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:29:54.197148 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:29:54.197929 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:29:54.197929 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:29:54.197929 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:29:54.197929 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:29:54.197929 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:29:54.197929 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:29:54.197929 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:29:54.197929 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:29:54.197929 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:29:54.197929 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:29:54.197929 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 17:29:54.197929 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:29:54.197929 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 98/201] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jun 16 17:29:53 2022] GET /identity/v3/roles/reader => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:29:54.205640 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33da2bab-fc6c-4384-ae65-df0092779b49 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:54.213488 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144bd708-79eb-4326-a0b3-25c337d85856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:54.214391 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144bd708-79eb-4326-a0b3-25c337d85856 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:54.214680 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144bd708-79eb-4326-a0b3-25c337d85856 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:54.214957 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144bd708-79eb-4326-a0b3-25c337d85856 None admin] PATH_INFO: `/v3/projects/d820cdcb6c0741e380190e6fe127e992` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:54.217314 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144bd708-79eb-4326-a0b3-25c337d85856 None admin] RBAC: Authorizing `identity:get_project(project_id=d820cdcb6c0741e380190e6fe127e992)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:54.218961 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144bd708-79eb-4326-a0b3-25c337d85856 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:54.223137 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 104/202] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jun 16 17:29:54 2022] GET /identity/v3/projects/d820cdcb6c0741e380190e6fe127e992 => generated 302 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:29:54.229586 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144bd708-79eb-4326-a0b3-25c337d85856 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:54.270425 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:54.331186 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33da2bab-fc6c-4384-ae65-df0092779b49 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:54.363510 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144bd708-79eb-4326-a0b3-25c337d85856 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:54.369528 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d762168-2674-4bfb-8ba7-a5a4c805c83b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:54.370517 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d762168-2674-4bfb-8ba7-a5a4c805c83b None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:54.371034 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d762168-2674-4bfb-8ba7-a5a4c805c83b None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:54.371595 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d762168-2674-4bfb-8ba7-a5a4c805c83b None admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:54.373280 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d762168-2674-4bfb-8ba7-a5a4c805c83b None admin] RBAC: Adding query filter params (name=reader) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:29:54.373805 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d762168-2674-4bfb-8ba7-a5a4c805c83b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:54.375598 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d762168-2674-4bfb-8ba7-a5a4c805c83b None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:54.400295 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 99/203] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Jun 16 17:29:54 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:29:54.404073 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec501d9-5870-41c6-8bf4-1bc9600cee08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:54.405619 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d762168-2674-4bfb-8ba7-a5a4c805c83b None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:54.406137 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec501d9-5870-41c6-8bf4-1bc9600cee08 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:54.406469 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec501d9-5870-41c6-8bf4-1bc9600cee08 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:54.410465 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec501d9-5870-41c6-8bf4-1bc9600cee08 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:54.411405 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec501d9-5870-41c6-8bf4-1bc9600cee08 None admin] RBAC: Adding query filter params (role.id=243fd0a11d94477986deb1dcfb3f31dd, scope.project.id=d820cdcb6c0741e380190e6fe127e992, user.id=bc760a5800b649989e800249b203636f) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:29:54.411725 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec501d9-5870-41c6-8bf4-1bc9600cee08 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:54.413251 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec501d9-5870-41c6-8bf4-1bc9600cee08 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:54.420464 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 105/204] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jun 16 17:29:54 2022] GET /identity/v3/role_assignments?user.id=bc760a5800b649989e800249b203636f&scope.project.id=d820cdcb6c0741e380190e6fe127e992&role.id=243fd0a11d94477986deb1dcfb3f31dd => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:29:54.542729 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d762168-2674-4bfb-8ba7-a5a4c805c83b None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:54.554664 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:54.599316 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df09b9d-c578-4f82-badb-2ca74f81fec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:54.600573 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df09b9d-c578-4f82-badb-2ca74f81fec4 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:54.601083 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df09b9d-c578-4f82-badb-2ca74f81fec4 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:54.601507 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df09b9d-c578-4f82-badb-2ca74f81fec4 None admin] PATH_INFO: `/v3/users/ebd6465c9b4f4f4aaed32bb07032d7f0` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:54.602950 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df09b9d-c578-4f82-badb-2ca74f81fec4 None admin] RBAC: Authorizing `identity:get_user(user_id=ebd6465c9b4f4f4aaed32bb07032d7f0)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:54.604599 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df09b9d-c578-4f82-badb-2ca74f81fec4 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:54.606906 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 100/205] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jun 16 17:29:54 2022] GET /identity/v3/users/ebd6465c9b4f4f4aaed32bb07032d7f0 => generated 289 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:29:54.612312 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec501d9-5870-41c6-8bf4-1bc9600cee08 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:54.648741 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:54.749057 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec501d9-5870-41c6-8bf4-1bc9600cee08 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:54.787637 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfdb1c6-8487-4ce1-8a2f-e0d3999e7a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:54.788576 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfdb1c6-8487-4ce1-8a2f-e0d3999e7a79 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:54.789033 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfdb1c6-8487-4ce1-8a2f-e0d3999e7a79 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:54.789460 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfdb1c6-8487-4ce1-8a2f-e0d3999e7a79 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:54.864768 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfdb1c6-8487-4ce1-8a2f-e0d3999e7a79 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:54.866480 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfdb1c6-8487-4ce1-8a2f-e0d3999e7a79 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:54.885804 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:54.978633 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 106/206] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jun 16 17:29:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:29:54.986238 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df09b9d-c578-4f82-badb-2ca74f81fec4 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:55.108730 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df09b9d-c578-4f82-badb-2ca74f81fec4 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:55.162134 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b18376f-74e3-453f-8b2b-fbabb6e91376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:55.163078 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b18376f-74e3-453f-8b2b-fbabb6e91376 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:55.163369 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b18376f-74e3-453f-8b2b-fbabb6e91376 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:55.163648 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b18376f-74e3-453f-8b2b-fbabb6e91376 None admin] PATH_INFO: `/v3/projects/3be66087f3954ee4bac1dc53ea4d59ed` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:55.164959 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b18376f-74e3-453f-8b2b-fbabb6e91376 None admin] RBAC: Authorizing `identity:get_project(project_id=3be66087f3954ee4bac1dc53ea4d59ed)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:55.166426 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b18376f-74e3-453f-8b2b-fbabb6e91376 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:55.168480 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 101/207] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jun 16 17:29:54 2022] GET /identity/v3/projects/3be66087f3954ee4bac1dc53ea4d59ed => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:29:55.174600 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfdb1c6-8487-4ce1-8a2f-e0d3999e7a79 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:55.291403 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfdb1c6-8487-4ce1-8a2f-e0d3999e7a79 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:55.308957 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:55.328662 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d0b15b-6316-41d5-9734-60169b3b5aff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:55.329254 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d0b15b-6316-41d5-9734-60169b3b5aff None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:55.330333 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d0b15b-6316-41d5-9734-60169b3b5aff None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:55.331037 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d0b15b-6316-41d5-9734-60169b3b5aff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:55.331989 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d0b15b-6316-41d5-9734-60169b3b5aff None admin] RBAC: Adding query filter params (role.id=9ed5382e55d649faa9195c4a9eb6fa57, scope.project.id=3be66087f3954ee4bac1dc53ea4d59ed, user.id=ebd6465c9b4f4f4aaed32bb07032d7f0) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:29:55.332335 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d0b15b-6316-41d5-9734-60169b3b5aff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:55.333895 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d0b15b-6316-41d5-9734-60169b3b5aff None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:55.340333 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 107/208] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jun 16 17:29:55 2022] GET /identity/v3/role_assignments?user.id=ebd6465c9b4f4f4aaed32bb07032d7f0&scope.project.id=3be66087f3954ee4bac1dc53ea4d59ed&role.id=9ed5382e55d649faa9195c4a9eb6fa57 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:29:56.263241 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f8cd36-6f80-485f-8bb9-cd33bf08a50a None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:56.263773 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f8cd36-6f80-485f-8bb9-cd33bf08a50a None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:56.263849 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f8cd36-6f80-485f-8bb9-cd33bf08a50a None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:56.265238 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 102/209] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:29:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:29:56.271010 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5d8ed6-c11a-474a-9f9e-687aefc8d1f7 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:56.271443 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5d8ed6-c11a-474a-9f9e-687aefc8d1f7 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:56.271832 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5d8ed6-c11a-474a-9f9e-687aefc8d1f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:56.290477 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-4e5d8ed6-c11a-474a-9f9e-687aefc8d1f7 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:29:56.291699 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-4e5d8ed6-c11a-474a-9f9e-687aefc8d1f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:29:56.364977 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 108/210] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:29:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:29:56.388229 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f8cd36-6f80-485f-8bb9-cd33bf08a50a None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:56.444182 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:56.467127 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c2c787-8d5f-45fe-afcc-bcd788fe34e4 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:56.467423 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c2c787-8d5f-45fe-afcc-bcd788fe34e4 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:56.467828 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c2c787-8d5f-45fe-afcc-bcd788fe34e4 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:56.469496 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 109/211] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:29:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:29:56.475129 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dc5a43-8843-425e-af84-aad5ebbaee7f None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:56.475414 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dc5a43-8843-425e-af84-aad5ebbaee7f None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:56.475699 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dc5a43-8843-425e-af84-aad5ebbaee7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:56.494121 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-82dc5a43-8843-425e-af84-aad5ebbaee7f None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:29:56.495354 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-82dc5a43-8843-425e-af84-aad5ebbaee7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:29:56.513718 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f8cd36-6f80-485f-8bb9-cd33bf08a50a None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:56.563949 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d033486-5575-4011-bf26-65592f30a475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:56.565421 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d033486-5575-4011-bf26-65592f30a475 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:56.565913 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d033486-5575-4011-bf26-65592f30a475 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:56.566586 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d033486-5575-4011-bf26-65592f30a475 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:56.571805 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 110/212] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:29:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:29:56.579681 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d033486-5575-4011-bf26-65592f30a475 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:29:56.587678 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-2d033486-5575-4011-bf26-65592f30a475 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:29:56.587678 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:29:56.587678 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:29:56.587678 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:29:56.587678 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:29:56.587678 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:29:56.587678 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:29:56.587678 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:29:56.587678 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:29:56.587678 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:29:56.587678 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:29:56.587678 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:29:56.587678 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:29:56.587678 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:29:56.587678 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:29:56.587678 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:29:56.588605 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:29:56.588605 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:29:56.588605 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:29:56.588605 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:29:56.588605 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:29:56.588605 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:29:56.588605 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:29:56.588605 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:29:56.588605 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:29:56.588605 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:29:56.588605 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:29:56.588605 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:29:56.588605 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:29:56.588605 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:29:56.588605 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:29:56.588605 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:29:56.589394 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:29:56.589394 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:29:56.589394 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:29:56.589394 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:29:56.589394 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:29:56.589394 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:29:56.589394 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:29:56.589394 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:29:56.589394 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:29:56.589394 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:29:56.589394 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:29:56.589394 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:29:56.591917 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 103/213] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Jun 16 17:29:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:29:56.597140 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dc5a43-8843-425e-af84-aad5ebbaee7f None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:56.598757 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d033486-5575-4011-bf26-65592f30a475 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:56.645139 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:56.711131 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dc5a43-8843-425e-af84-aad5ebbaee7f None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:56.751739 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190d2cd6-b4eb-4341-83d0-f760ffc3dafc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:56.752691 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190d2cd6-b4eb-4341-83d0-f760ffc3dafc None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:56.753038 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190d2cd6-b4eb-4341-83d0-f760ffc3dafc None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:56.753381 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190d2cd6-b4eb-4341-83d0-f760ffc3dafc None admin] PATH_INFO: `/v3/roles/member` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:56.753554 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d033486-5575-4011-bf26-65592f30a475 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:56.758161 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:56.770028 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190d2cd6-b4eb-4341-83d0-f760ffc3dafc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:29:56.780803 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-190d2cd6-b4eb-4341-83d0-f760ffc3dafc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:29:56.780803 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:29:56.780803 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:29:56.780803 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:29:56.780803 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:29:56.780803 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:29:56.780803 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:29:56.780803 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:29:56.780803 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:29:56.780803 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:29:56.780803 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:29:56.780803 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:29:56.780803 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:29:56.780803 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:29:56.780803 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:29:56.780803 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:29:56.782332 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:29:56.782332 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:29:56.782332 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:29:56.782332 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:29:56.782332 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:29:56.782332 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:29:56.782332 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:29:56.782332 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:29:56.782332 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:29:56.782332 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:29:56.782332 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:29:56.782332 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:29:56.782332 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:29:56.782332 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:29:56.782332 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:29:56.782332 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:29:56.783101 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:29:56.783101 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:29:56.783101 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:29:56.783101 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:29:56.783101 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:29:56.783101 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:29:56.783101 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:29:56.783101 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:29:56.783101 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:29:56.783101 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:29:56.783101 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:29:56.783101 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:29:56.785851 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 111/214] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jun 16 17:29:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:29:56.791539 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190d2cd6-b4eb-4341-83d0-f760ffc3dafc None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:56.805173 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57819dcf-6ea3-4447-94c0-54c81c35ff8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:56.806142 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57819dcf-6ea3-4447-94c0-54c81c35ff8a None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:56.807162 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57819dcf-6ea3-4447-94c0-54c81c35ff8a None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:56.807659 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57819dcf-6ea3-4447-94c0-54c81c35ff8a None admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:56.809149 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57819dcf-6ea3-4447-94c0-54c81c35ff8a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:29:56.809802 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57819dcf-6ea3-4447-94c0-54c81c35ff8a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:56.811849 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57819dcf-6ea3-4447-94c0-54c81c35ff8a None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:56.824439 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 104/215] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jun 16 17:29:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:29:56.830501 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57819dcf-6ea3-4447-94c0-54c81c35ff8a None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:56.919532 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:56.924728 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190d2cd6-b4eb-4341-83d0-f760ffc3dafc None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:56.954626 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57819dcf-6ea3-4447-94c0-54c81c35ff8a None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:56.963121 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613bf5ca-2070-415c-a494-4fd58b147763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:56.963802 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613bf5ca-2070-415c-a494-4fd58b147763 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:56.964110 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613bf5ca-2070-415c-a494-4fd58b147763 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:56.964425 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613bf5ca-2070-415c-a494-4fd58b147763 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:56.965084 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613bf5ca-2070-415c-a494-4fd58b147763 None admin] RBAC: Adding query filter params (name=member) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:29:56.965384 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613bf5ca-2070-415c-a494-4fd58b147763 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:56.966840 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613bf5ca-2070-415c-a494-4fd58b147763 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:56.981534 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 112/216] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Jun 16 17:29:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:29:56.982722 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac2de75-734e-4f8f-bee0-b29a1638d540 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:56.983012 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac2de75-734e-4f8f-bee0-b29a1638d540 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:56.983290 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac2de75-734e-4f8f-bee0-b29a1638d540 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:56.985345 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 113/217] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:29:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:29:56.986206 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac2de75-734e-4f8f-bee0-b29a1638d540 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:57.000755 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb45bd1-d836-4f1a-931e-f9772df5e89a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:57.001758 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb45bd1-d836-4f1a-931e-f9772df5e89a None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:57.002045 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb45bd1-d836-4f1a-931e-f9772df5e89a None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:57.002324 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb45bd1-d836-4f1a-931e-f9772df5e89a None admin] PATH_INFO: `/v3/users/bc760a5800b649989e800249b203636f` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:57.003634 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb45bd1-d836-4f1a-931e-f9772df5e89a None admin] RBAC: Authorizing `identity:get_user(user_id=bc760a5800b649989e800249b203636f)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:57.005183 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb45bd1-d836-4f1a-931e-f9772df5e89a None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:57.006839 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 105/218] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jun 16 17:29:56 2022] GET /identity/v3/users/bc760a5800b649989e800249b203636f => generated 283 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:29:57.008141 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57953c1c-072f-4aff-a6ea-2f747a9f7845 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:57.008459 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57953c1c-072f-4aff-a6ea-2f747a9f7845 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:57.008740 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57953c1c-072f-4aff-a6ea-2f747a9f7845 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:57.027817 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-57953c1c-072f-4aff-a6ea-2f747a9f7845 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:29:57.028957 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-57953c1c-072f-4aff-a6ea-2f747a9f7845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:29:57.064943 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:57.112417 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 106/219] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:29:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:29:57.117118 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac2de75-734e-4f8f-bee0-b29a1638d540 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:57.117535 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57953c1c-072f-4aff-a6ea-2f747a9f7845 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:57.156726 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2f820d-5cca-486f-88b9-534fde42e56f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:57.157689 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2f820d-5cca-486f-88b9-534fde42e56f None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:57.157976 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2f820d-5cca-486f-88b9-534fde42e56f None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:57.158250 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2f820d-5cca-486f-88b9-534fde42e56f None admin] PATH_INFO: `/v3/users/f49bf79057404437b9ab776a195d6339` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:57.173030 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2f820d-5cca-486f-88b9-534fde42e56f None admin] RBAC: Authorizing `identity:get_user(user_id=f49bf79057404437b9ab776a195d6339)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:57.174567 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2f820d-5cca-486f-88b9-534fde42e56f None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:57.176315 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 114/220] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jun 16 17:29:56 2022] GET /identity/v3/users/f49bf79057404437b9ab776a195d6339 => generated 297 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:29:57.177165 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2f820d-5cca-486f-88b9-534fde42e56f None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:57.202805 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:57.226710 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57953c1c-072f-4aff-a6ea-2f747a9f7845 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:57.271100 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1087182a-956a-47e4-9b5a-27cec43ffe64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:57.271988 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1087182a-956a-47e4-9b5a-27cec43ffe64 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:57.272280 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1087182a-956a-47e4-9b5a-27cec43ffe64 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:57.272589 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1087182a-956a-47e4-9b5a-27cec43ffe64 None admin] PATH_INFO: `/v3/projects/d820cdcb6c0741e380190e6fe127e992` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:57.273834 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1087182a-956a-47e4-9b5a-27cec43ffe64 None admin] RBAC: Authorizing `identity:get_project(project_id=d820cdcb6c0741e380190e6fe127e992)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:57.275357 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1087182a-956a-47e4-9b5a-27cec43ffe64 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:57.277129 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 107/221] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jun 16 17:29:57 2022] GET /identity/v3/projects/d820cdcb6c0741e380190e6fe127e992 => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:29:57.278393 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7127b7b7-3b51-4ec7-af78-4ce5bc0d3c20 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:57.278628 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7127b7b7-3b51-4ec7-af78-4ce5bc0d3c20 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:57.278906 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7127b7b7-3b51-4ec7-af78-4ce5bc0d3c20 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:57.280148 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 108/222] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:29:57 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:29:57.280883 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7127b7b7-3b51-4ec7-af78-4ce5bc0d3c20 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:57.292975 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2f820d-5cca-486f-88b9-534fde42e56f None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:57.314469 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:57.328479 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d7c26c-518d-4295-8f4e-e17ebfd9ef19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:57.329415 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d7c26c-518d-4295-8f4e-e17ebfd9ef19 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:57.329692 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d7c26c-518d-4295-8f4e-e17ebfd9ef19 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:57.329966 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d7c26c-518d-4295-8f4e-e17ebfd9ef19 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:57.331132 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d7c26c-518d-4295-8f4e-e17ebfd9ef19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:57.332685 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d7c26c-518d-4295-8f4e-e17ebfd9ef19 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:57.334488 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 115/223] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jun 16 17:29:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:29:57.335032 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d7c26c-518d-4295-8f4e-e17ebfd9ef19 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:57.400706 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7127b7b7-3b51-4ec7-af78-4ce5bc0d3c20 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:57.424580 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:57.437464 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d7c26c-518d-4295-8f4e-e17ebfd9ef19 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:57.439511 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84a1b84-cc47-45db-93e1-e39857dd23b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:57.440165 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84a1b84-cc47-45db-93e1-e39857dd23b7 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:57.440479 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84a1b84-cc47-45db-93e1-e39857dd23b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:57.440757 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84a1b84-cc47-45db-93e1-e39857dd23b7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:57.472927 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db914be-3f7f-4bb1-a9ac-7b081a3e26f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:57.473817 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db914be-3f7f-4bb1-a9ac-7b081a3e26f6 None admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:57.474097 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db914be-3f7f-4bb1-a9ac-7b081a3e26f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:57.474372 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db914be-3f7f-4bb1-a9ac-7b081a3e26f6 None admin] PATH_INFO: `/v3/projects/d820cdcb6c0741e380190e6fe127e992/users/bc760a5800b649989e800249b203636f/roles/3ca2dcbbd29d4a31a3ab78f2299bc239` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:57.476327 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db914be-3f7f-4bb1-a9ac-7b081a3e26f6 None admin] RBAC: Authorizing `identity:create_grant(project_id=d820cdcb6c0741e380190e6fe127e992, user_id=bc760a5800b649989e800249b203636f, role_id=3ca2dcbbd29d4a31a3ab78f2299bc239)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:57.477904 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db914be-3f7f-4bb1-a9ac-7b081a3e26f6 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:57.490751 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 116/224] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Jun 16 17:29:57 2022] PUT /identity/v3/projects/d820cdcb6c0741e380190e6fe127e992/users/bc760a5800b649989e800249b203636f/roles/3ca2dcbbd29d4a31a3ab78f2299bc239 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:29:57.493671 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4693f8b1-1ca0-4604-b3bf-8dc24a1937fc None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:57.493958 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4693f8b1-1ca0-4604-b3bf-8dc24a1937fc None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:57.494237 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4693f8b1-1ca0-4604-b3bf-8dc24a1937fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:57.514748 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84a1b84-cc47-45db-93e1-e39857dd23b7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:57.515258 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-4693f8b1-1ca0-4604-b3bf-8dc24a1937fc None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:29:57.516920 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84a1b84-cc47-45db-93e1-e39857dd23b7 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:57.517820 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-4693f8b1-1ca0-4604-b3bf-8dc24a1937fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:29:57.553601 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:57.606212 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 117/225] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:29:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:29:57.606212 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4693f8b1-1ca0-4604-b3bf-8dc24a1937fc None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:57.638677 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 109/226] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jun 16 17:29:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:29:57.641871 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84a1b84-cc47-45db-93e1-e39857dd23b7 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:57.708587 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:57.737305 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4693f8b1-1ca0-4604-b3bf-8dc24a1937fc None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:57.753834 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84a1b84-cc47-45db-93e1-e39857dd23b7 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:57.773388 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6370710-33ad-47e8-8252-fe0d8809d456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:57.774067 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6370710-33ad-47e8-8252-fe0d8809d456 None admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:57.774346 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6370710-33ad-47e8-8252-fe0d8809d456 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:57.774620 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6370710-33ad-47e8-8252-fe0d8809d456 None admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:57.775325 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6370710-33ad-47e8-8252-fe0d8809d456 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:57.776794 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6370710-33ad-47e8-8252-fe0d8809d456 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:57.786084 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:57.791877 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e329393-0910-442f-9a14-98268b21b125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:57.792743 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e329393-0910-442f-9a14-98268b21b125 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:57.793025 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e329393-0910-442f-9a14-98268b21b125 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:57.793295 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e329393-0910-442f-9a14-98268b21b125 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:57.805152 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e329393-0910-442f-9a14-98268b21b125 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:29:57.808028 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 118/227] 10.222.0.42 () {60 vars in 1132 bytes} [Thu Jun 16 17:29:57 2022] POST /identity/v3/users => generated 297 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:29:57.809300 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6370710-33ad-47e8-8252-fe0d8809d456 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:57.814046 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-1e329393-0910-442f-9a14-98268b21b125 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 17:29:57.814046 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:29:57.814046 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:29:57.814046 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:29:57.814046 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:29:57.814046 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:29:57.814046 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:29:57.814046 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:29:57.814046 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:29:57.814046 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:29:57.814046 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:29:57.814046 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:29:57.814046 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:29:57.814046 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:29:57.814046 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:29:57.814046 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:29:57.815038 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:29:57.815038 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:29:57.815038 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:29:57.815038 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:29:57.815038 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:29:57.815038 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:29:57.815038 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:29:57.815038 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:29:57.815038 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:29:57.815038 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:29:57.815038 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:29:57.815038 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:29:57.815038 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:29:57.815038 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:29:57.815038 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:29:57.815038 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:29:57.815856 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:29:57.815856 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:29:57.815856 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:29:57.815856 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:29:57.815856 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:29:57.815856 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:29:57.815856 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:29:57.815856 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:29:57.815856 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:29:57.815856 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:29:57.815856 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 17:29:57.815856 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:29:57.815856 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 110/228] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Jun 16 17:29:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:29:57.823043 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e329393-0910-442f-9a14-98268b21b125 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:57.948016 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6370710-33ad-47e8-8252-fe0d8809d456 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:57.989642 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e329393-0910-442f-9a14-98268b21b125 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:57.992996 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f6f3da-c971-4583-b20f-c3b57dcb5492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:57.994809 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f6f3da-c971-4583-b20f-c3b57dcb5492 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:57.995097 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f6f3da-c971-4583-b20f-c3b57dcb5492 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:57.995378 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f6f3da-c971-4583-b20f-c3b57dcb5492 None admin] PATH_INFO: `/v3/projects/d820cdcb6c0741e380190e6fe127e992` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:57.996691 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f6f3da-c971-4583-b20f-c3b57dcb5492 None admin] RBAC: Authorizing `identity:get_project(project_id=d820cdcb6c0741e380190e6fe127e992)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:57.998185 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f6f3da-c971-4583-b20f-c3b57dcb5492 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:58.002316 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 119/229] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jun 16 17:29:57 2022] GET /identity/v3/projects/d820cdcb6c0741e380190e6fe127e992 => generated 302 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:29:58.008902 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f6f3da-c971-4583-b20f-c3b57dcb5492 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:58.026569 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:58.039310 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba0c5e0-9643-43b7-bc8e-fc91d45beea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:58.040125 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba0c5e0-9643-43b7-bc8e-fc91d45beea1 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:58.040348 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba0c5e0-9643-43b7-bc8e-fc91d45beea1 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:58.041990 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba0c5e0-9643-43b7-bc8e-fc91d45beea1 None admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:58.041990 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba0c5e0-9643-43b7-bc8e-fc91d45beea1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:29:58.041990 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba0c5e0-9643-43b7-bc8e-fc91d45beea1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:58.043516 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba0c5e0-9643-43b7-bc8e-fc91d45beea1 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:58.053971 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 111/230] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jun 16 17:29:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:29:58.059578 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba0c5e0-9643-43b7-bc8e-fc91d45beea1 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:58.126587 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:58.154452 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f6f3da-c971-4583-b20f-c3b57dcb5492 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:58.198851 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba0c5e0-9643-43b7-bc8e-fc91d45beea1 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:58.204676 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15821655-b0ed-43ac-82a6-d0a74e78183d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:58.205511 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15821655-b0ed-43ac-82a6-d0a74e78183d None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:58.208794 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15821655-b0ed-43ac-82a6-d0a74e78183d None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:58.209284 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15821655-b0ed-43ac-82a6-d0a74e78183d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:58.210403 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15821655-b0ed-43ac-82a6-d0a74e78183d None admin] RBAC: Adding query filter params (role.id=0be2075946f54c8485f1c275f746478a, scope.project.id=d820cdcb6c0741e380190e6fe127e992, user.id=f49bf79057404437b9ab776a195d6339) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:29:58.210863 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15821655-b0ed-43ac-82a6-d0a74e78183d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:58.212576 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15821655-b0ed-43ac-82a6-d0a74e78183d None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:58.221544 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 120/231] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jun 16 17:29:58 2022] GET /identity/v3/role_assignments?user.id=f49bf79057404437b9ab776a195d6339&scope.project.id=d820cdcb6c0741e380190e6fe127e992&role.id=0be2075946f54c8485f1c275f746478a => generated 259 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:29:58.246909 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2f71a8-163e-4214-a7f9-559711967d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:58.247868 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2f71a8-163e-4214-a7f9-559711967d7a None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:58.248161 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2f71a8-163e-4214-a7f9-559711967d7a None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:58.248534 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2f71a8-163e-4214-a7f9-559711967d7a None admin] PATH_INFO: `/v3/users/bc760a5800b649989e800249b203636f` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:58.249916 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2f71a8-163e-4214-a7f9-559711967d7a None admin] RBAC: Authorizing `identity:get_user(user_id=bc760a5800b649989e800249b203636f)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:58.252049 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2f71a8-163e-4214-a7f9-559711967d7a None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:58.253923 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 112/232] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jun 16 17:29:58 2022] GET /identity/v3/users/bc760a5800b649989e800249b203636f => generated 283 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:29:58.260074 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15821655-b0ed-43ac-82a6-d0a74e78183d None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:58.346918 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:58.403894 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15821655-b0ed-43ac-82a6-d0a74e78183d None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:58.448773 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c609181-f171-4dbd-be4c-4f21b286ef01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:58.449638 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c609181-f171-4dbd-be4c-4f21b286ef01 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:58.449919 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c609181-f171-4dbd-be4c-4f21b286ef01 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:58.450197 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c609181-f171-4dbd-be4c-4f21b286ef01 None admin] PATH_INFO: `/v3/projects/d820cdcb6c0741e380190e6fe127e992` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:58.452031 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c609181-f171-4dbd-be4c-4f21b286ef01 None admin] RBAC: Authorizing `identity:get_project(project_id=d820cdcb6c0741e380190e6fe127e992)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:58.453593 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c609181-f171-4dbd-be4c-4f21b286ef01 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:58.456230 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 121/233] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jun 16 17:29:58 2022] GET /identity/v3/projects/d820cdcb6c0741e380190e6fe127e992 => generated 302 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:29:58.461694 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2f71a8-163e-4214-a7f9-559711967d7a None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:58.587477 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2f71a8-163e-4214-a7f9-559711967d7a None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:58.612028 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:58.623955 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142bf218-12d0-4ea1-9350-88dc239f9b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:58.624904 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142bf218-12d0-4ea1-9350-88dc239f9b22 None admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:58.625635 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142bf218-12d0-4ea1-9350-88dc239f9b22 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:58.625930 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142bf218-12d0-4ea1-9350-88dc239f9b22 None admin] PATH_INFO: `/v3/projects/d820cdcb6c0741e380190e6fe127e992/users/bc760a5800b649989e800249b203636f/roles/243fd0a11d94477986deb1dcfb3f31dd` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:58.628254 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142bf218-12d0-4ea1-9350-88dc239f9b22 None admin] RBAC: Authorizing `identity:create_grant(project_id=d820cdcb6c0741e380190e6fe127e992, user_id=bc760a5800b649989e800249b203636f, role_id=243fd0a11d94477986deb1dcfb3f31dd)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:58.630262 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142bf218-12d0-4ea1-9350-88dc239f9b22 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:58.646009 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 113/234] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Jun 16 17:29:58 2022] PUT /identity/v3/projects/d820cdcb6c0741e380190e6fe127e992/users/bc760a5800b649989e800249b203636f/roles/243fd0a11d94477986deb1dcfb3f31dd => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:30:00.123747 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb93404-5779-4b27-a153-734317b0c37d None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:00.124280 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb93404-5779-4b27-a153-734317b0c37d None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:00.124280 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb93404-5779-4b27-a153-734317b0c37d None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:00.126366 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 122/235] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:30:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:30:00.131806 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed03ae30-4dd8-4e4f-8143-6b1105a72cbd None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:00.132238 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed03ae30-4dd8-4e4f-8143-6b1105a72cbd None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:00.132435 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed03ae30-4dd8-4e4f-8143-6b1105a72cbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:00.157739 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-ed03ae30-4dd8-4e4f-8143-6b1105a72cbd None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:30:00.158869 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-ed03ae30-4dd8-4e4f-8143-6b1105a72cbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:30:00.286154 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 114/236] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:30:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:30:00.328199 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb93404-5779-4b27-a153-734317b0c37d None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:00.467083 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb93404-5779-4b27-a153-734317b0c37d None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:00.517706 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40580f7c-bcf5-4bed-820b-f1162adbfda3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:00.518600 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40580f7c-bcf5-4bed-820b-f1162adbfda3 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:00.518895 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40580f7c-bcf5-4bed-820b-f1162adbfda3 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:00.519169 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40580f7c-bcf5-4bed-820b-f1162adbfda3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:00.521063 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:00.533268 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40580f7c-bcf5-4bed-820b-f1162adbfda3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:30:00.540999 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-40580f7c-bcf5-4bed-820b-f1162adbfda3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:30:00.540999 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:30:00.540999 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:30:00.540999 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:30:00.540999 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:30:00.540999 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:30:00.540999 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:30:00.540999 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:30:00.540999 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:30:00.540999 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:30:00.540999 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:30:00.540999 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:30:00.540999 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:30:00.540999 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:30:00.540999 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:30:00.540999 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:30:00.541892 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:30:00.541892 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:30:00.541892 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:30:00.541892 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:30:00.541892 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:30:00.541892 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:30:00.541892 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:30:00.541892 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:30:00.541892 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:30:00.541892 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:30:00.541892 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:30:00.541892 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:30:00.541892 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:30:00.541892 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:30:00.541892 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:30:00.541892 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:30:00.542688 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:30:00.542688 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:30:00.542688 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:30:00.542688 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:30:00.542688 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:30:00.542688 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:30:00.542688 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:30:00.542688 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:30:00.542688 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:30:00.542688 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:30:00.542688 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:30:00.542688 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:30:00.544730 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 123/237] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Jun 16 17:30:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:30:00.549715 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed03ae30-4dd8-4e4f-8143-6b1105a72cbd None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:00.695602 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed03ae30-4dd8-4e4f-8143-6b1105a72cbd None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:00.711461 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a567e7-7c98-4528-a54a-037763693734 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:00.711771 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a567e7-7c98-4528-a54a-037763693734 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:00.712017 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a567e7-7c98-4528-a54a-037763693734 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:00.713488 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 124/238] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:30:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:30:00.718804 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:00.720598 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ce0629-1b0b-4417-b233-ffca977706dd None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:00.720930 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ce0629-1b0b-4417-b233-ffca977706dd None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:00.721876 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ce0629-1b0b-4417-b233-ffca977706dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:00.745439 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-c4ce0629-1b0b-4417-b233-ffca977706dd None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:30:00.746656 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-c4ce0629-1b0b-4417-b233-ffca977706dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:30:00.767937 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37243ad7-0156-4b73-83a1-abd7eaf50c4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:00.768748 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37243ad7-0156-4b73-83a1-abd7eaf50c4f None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:00.769039 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37243ad7-0156-4b73-83a1-abd7eaf50c4f None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:00.769315 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37243ad7-0156-4b73-83a1-abd7eaf50c4f None admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:00.770083 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37243ad7-0156-4b73-83a1-abd7eaf50c4f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:00.770392 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37243ad7-0156-4b73-83a1-abd7eaf50c4f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:00.772598 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37243ad7-0156-4b73-83a1-abd7eaf50c4f None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:00.784693 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 115/239] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jun 16 17:30:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:00.786376 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fe5b3c-bf94-4a4f-b8d8-386d0b9570d8 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:00.786674 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fe5b3c-bf94-4a4f-b8d8-386d0b9570d8 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:00.786954 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fe5b3c-bf94-4a4f-b8d8-386d0b9570d8 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:00.788509 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 116/240] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:30:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:30:00.793293 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fe5b3c-bf94-4a4f-b8d8-386d0b9570d8 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:00.836159 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 125/241] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:30:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 17:30:00.837935 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b53ed9d-e4cf-4257-bb66-ed69e4399cb2 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:00.838236 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b53ed9d-e4cf-4257-bb66-ed69e4399cb2 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:00.839463 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b53ed9d-e4cf-4257-bb66-ed69e4399cb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:00.860701 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-6b53ed9d-e4cf-4257-bb66-ed69e4399cb2 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:30:00.864127 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-6b53ed9d-e4cf-4257-bb66-ed69e4399cb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:30:00.913253 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fe5b3c-bf94-4a4f-b8d8-386d0b9570d8 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:00.950446 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23289693-b823-41bd-831a-c3e91021cd95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:00.951522 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23289693-b823-41bd-831a-c3e91021cd95 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:00.952022 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23289693-b823-41bd-831a-c3e91021cd95 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:00.952445 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23289693-b823-41bd-831a-c3e91021cd95 None admin] PATH_INFO: `/v3/users/bc760a5800b649989e800249b203636f` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:00.953310 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 126/242] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:30:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:30:00.954186 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23289693-b823-41bd-831a-c3e91021cd95 None admin] RBAC: Authorizing `identity:get_user(user_id=bc760a5800b649989e800249b203636f)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:00.954307 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b53ed9d-e4cf-4257-bb66-ed69e4399cb2 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:00.956103 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23289693-b823-41bd-831a-c3e91021cd95 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:00.957956 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 117/243] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jun 16 17:30:00 2022] GET /identity/v3/users/bc760a5800b649989e800249b203636f => generated 283 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:00.965747 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23289693-b823-41bd-831a-c3e91021cd95 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:01.030249 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:01.041087 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:01.112353 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b53ed9d-e4cf-4257-bb66-ed69e4399cb2 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:01.135981 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23289693-b823-41bd-831a-c3e91021cd95 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:01.186372 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0736e882-85a9-415a-ab62-7eb9c89a1eb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:01.187313 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c1a391-6bdc-461e-8687-496b908fb191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:01.188713 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0736e882-85a9-415a-ab62-7eb9c89a1eb7 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:01.188887 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c1a391-6bdc-461e-8687-496b908fb191 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:01.189655 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c1a391-6bdc-461e-8687-496b908fb191 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:01.189814 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0736e882-85a9-415a-ab62-7eb9c89a1eb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:01.189982 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c1a391-6bdc-461e-8687-496b908fb191 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:01.190366 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0736e882-85a9-415a-ab62-7eb9c89a1eb7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:01.193007 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0736e882-85a9-415a-ab62-7eb9c89a1eb7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:01.195955 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0736e882-85a9-415a-ab62-7eb9c89a1eb7 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:01.199580 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 127/244] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jun 16 17:30:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:01.201024 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0736e882-85a9-415a-ab62-7eb9c89a1eb7 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:01.282847 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c1a391-6bdc-461e-8687-496b908fb191 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:01.284230 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c1a391-6bdc-461e-8687-496b908fb191 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:01.333440 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:01.368026 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0736e882-85a9-415a-ab62-7eb9c89a1eb7 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:01.401847 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 118/245] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jun 16 17:30:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:30:01.402529 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c1a391-6bdc-461e-8687-496b908fb191 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:01.412409 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:01.426004 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bf9c9e-fd40-427c-afdc-9840ab7ec403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:01.426986 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bf9c9e-fd40-427c-afdc-9840ab7ec403 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:01.427264 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bf9c9e-fd40-427c-afdc-9840ab7ec403 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:01.427617 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bf9c9e-fd40-427c-afdc-9840ab7ec403 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:01.439683 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bf9c9e-fd40-427c-afdc-9840ab7ec403 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:30:01.447347 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-d3bf9c9e-fd40-427c-afdc-9840ab7ec403 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:30:01.447347 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:30:01.447347 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:30:01.447347 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:30:01.447347 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:30:01.447347 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:30:01.447347 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:30:01.447347 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:30:01.447347 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:30:01.447347 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:30:01.447347 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:30:01.447347 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:30:01.447347 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:30:01.447347 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:30:01.447347 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:30:01.447347 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:30:01.448406 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:30:01.448406 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:30:01.448406 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:30:01.448406 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:30:01.448406 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:30:01.448406 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:30:01.448406 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:30:01.448406 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:30:01.448406 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:30:01.448406 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:30:01.448406 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:30:01.448406 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:30:01.448406 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:30:01.448406 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:30:01.448406 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:30:01.448406 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:30:01.449312 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:30:01.449312 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:30:01.449312 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:30:01.449312 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:30:01.449312 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:30:01.449312 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:30:01.449312 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:30:01.449312 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:30:01.449312 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:30:01.449312 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:30:01.449312 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:30:01.449312 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:30:01.449312 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 128/246] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jun 16 17:30:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:30:01.450184 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123a8d0f-1007-4196-81c7-df0a083a634f None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:01.450184 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123a8d0f-1007-4196-81c7-df0a083a634f None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:01.451185 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123a8d0f-1007-4196-81c7-df0a083a634f None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:01.456506 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 129/247] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:30:01 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:30:01.456506 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123a8d0f-1007-4196-81c7-df0a083a634f None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:01.542040 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c1a391-6bdc-461e-8687-496b908fb191 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:01.582337 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6258101b-a282-40e1-89bd-6ac5d89750f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:01.583039 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6258101b-a282-40e1-89bd-6ac5d89750f0 None admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:01.583319 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6258101b-a282-40e1-89bd-6ac5d89750f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:01.583590 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6258101b-a282-40e1-89bd-6ac5d89750f0 None admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:01.584302 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6258101b-a282-40e1-89bd-6ac5d89750f0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:01.585769 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6258101b-a282-40e1-89bd-6ac5d89750f0 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:01.599755 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123a8d0f-1007-4196-81c7-df0a083a634f None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:01.600293 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:01.606754 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 119/248] 10.222.0.42 () {60 vars in 1132 bytes} [Thu Jun 16 17:30:01 2022] POST /identity/v3/users => generated 293 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:30:01.607689 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6258101b-a282-40e1-89bd-6ac5d89750f0 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:01.647848 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84606658-6a0c-4758-af56-ede7181534e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:01.649249 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84606658-6a0c-4758-af56-ede7181534e0 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:01.650609 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84606658-6a0c-4758-af56-ede7181534e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:01.651098 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84606658-6a0c-4758-af56-ede7181534e0 None admin] PATH_INFO: `/v3/projects/d820cdcb6c0741e380190e6fe127e992` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:01.652640 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84606658-6a0c-4758-af56-ede7181534e0 None admin] RBAC: Authorizing `identity:get_project(project_id=d820cdcb6c0741e380190e6fe127e992)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:01.654524 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84606658-6a0c-4758-af56-ede7181534e0 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:01.658249 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 130/249] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jun 16 17:30:01 2022] GET /identity/v3/projects/d820cdcb6c0741e380190e6fe127e992 => generated 302 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:01.660539 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e52dfd9-cedc-429a-9d21-97ad7fd97d53 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:01.661054 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e52dfd9-cedc-429a-9d21-97ad7fd97d53 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:01.661658 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e52dfd9-cedc-429a-9d21-97ad7fd97d53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:01.683214 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-4e52dfd9-cedc-429a-9d21-97ad7fd97d53 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:30:01.684509 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-4e52dfd9-cedc-429a-9d21-97ad7fd97d53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:30:01.727104 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6258101b-a282-40e1-89bd-6ac5d89750f0 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:01.737321 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:01.763376 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 131/250] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:30:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:30:01.768310 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e31787e-032a-4c72-bbfd-84e3e3247932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:01.769021 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e31787e-032a-4c72-bbfd-84e3e3247932 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:01.769304 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e31787e-032a-4c72-bbfd-84e3e3247932 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:01.769638 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e31787e-032a-4c72-bbfd-84e3e3247932 None admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:01.770393 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e31787e-032a-4c72-bbfd-84e3e3247932 None admin] RBAC: Adding query filter params (name=member) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:01.770749 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e31787e-032a-4c72-bbfd-84e3e3247932 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:01.772153 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e31787e-032a-4c72-bbfd-84e3e3247932 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:01.774393 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b28d8be-a549-4a4e-9a6c-cd934f26cb7a None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:01.774393 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b28d8be-a549-4a4e-9a6c-cd934f26cb7a None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:01.774588 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b28d8be-a549-4a4e-9a6c-cd934f26cb7a None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:01.776488 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 132/251] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:30:01 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:30:01.776611 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b28d8be-a549-4a4e-9a6c-cd934f26cb7a None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:01.782444 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 120/252] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Jun 16 17:30:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:01.783912 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136d1d4d-194b-4b9e-8d5e-2501690101d9 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:01.784089 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136d1d4d-194b-4b9e-8d5e-2501690101d9 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:01.784385 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136d1d4d-194b-4b9e-8d5e-2501690101d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:01.802482 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-136d1d4d-194b-4b9e-8d5e-2501690101d9 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:30:01.803599 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-136d1d4d-194b-4b9e-8d5e-2501690101d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:30:01.885633 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:01.909226 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 121/253] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:30:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:30:01.909226 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136d1d4d-194b-4b9e-8d5e-2501690101d9 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:01.937078 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b28d8be-a549-4a4e-9a6c-cd934f26cb7a None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:01.980650 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affdf06d-62d5-46ba-bcf2-0fc34d5e667e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:01.981323 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affdf06d-62d5-46ba-bcf2-0fc34d5e667e None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:01.981603 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affdf06d-62d5-46ba-bcf2-0fc34d5e667e None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:01.981875 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affdf06d-62d5-46ba-bcf2-0fc34d5e667e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:01.982708 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affdf06d-62d5-46ba-bcf2-0fc34d5e667e None admin] RBAC: Adding query filter params (role.id=3ca2dcbbd29d4a31a3ab78f2299bc239, scope.project.id=d820cdcb6c0741e380190e6fe127e992, user.id=bc760a5800b649989e800249b203636f) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:01.983019 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affdf06d-62d5-46ba-bcf2-0fc34d5e667e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:01.985017 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affdf06d-62d5-46ba-bcf2-0fc34d5e667e None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:01.992919 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 133/254] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jun 16 17:30:01 2022] GET /identity/v3/role_assignments?user.id=bc760a5800b649989e800249b203636f&scope.project.id=d820cdcb6c0741e380190e6fe127e992&role.id=3ca2dcbbd29d4a31a3ab78f2299bc239 => generated 608 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:01.993576 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affdf06d-62d5-46ba-bcf2-0fc34d5e667e None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:02.030908 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136d1d4d-194b-4b9e-8d5e-2501690101d9 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:02.059183 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:02.086385 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b637a97-1b69-4ab2-b890-2d5e6b8e6565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:02.087933 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b637a97-1b69-4ab2-b890-2d5e6b8e6565 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:02.088521 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b637a97-1b69-4ab2-b890-2d5e6b8e6565 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:02.088809 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b637a97-1b69-4ab2-b890-2d5e6b8e6565 None admin] PATH_INFO: `/v3/users/f49bf79057404437b9ab776a195d6339` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:02.090391 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b637a97-1b69-4ab2-b890-2d5e6b8e6565 None admin] RBAC: Authorizing `identity:get_user(user_id=f49bf79057404437b9ab776a195d6339)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:02.092989 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b637a97-1b69-4ab2-b890-2d5e6b8e6565 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:02.095481 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 122/255] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jun 16 17:30:01 2022] GET /identity/v3/users/f49bf79057404437b9ab776a195d6339 => generated 297 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:02.096315 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b637a97-1b69-4ab2-b890-2d5e6b8e6565 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:02.128029 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affdf06d-62d5-46ba-bcf2-0fc34d5e667e None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:02.184640 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:02.196860 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67a0055-5869-41b5-b77b-6d3d42c6c0a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:02.198471 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67a0055-5869-41b5-b77b-6d3d42c6c0a8 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:02.199009 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67a0055-5869-41b5-b77b-6d3d42c6c0a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:02.199507 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67a0055-5869-41b5-b77b-6d3d42c6c0a8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:02.213548 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67a0055-5869-41b5-b77b-6d3d42c6c0a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:30:02.222971 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-d67a0055-5869-41b5-b77b-6d3d42c6c0a8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 17:30:02.222971 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:30:02.222971 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:30:02.222971 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:30:02.222971 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:30:02.222971 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:30:02.222971 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:30:02.222971 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:30:02.222971 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:30:02.222971 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:30:02.222971 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:30:02.222971 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:30:02.222971 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:30:02.222971 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:30:02.222971 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:30:02.222971 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:30:02.223891 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:30:02.223891 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:30:02.223891 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:30:02.223891 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:30:02.223891 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:30:02.223891 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:30:02.223891 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:30:02.223891 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:30:02.223891 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:30:02.223891 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:30:02.223891 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:30:02.223891 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:30:02.223891 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:30:02.223891 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:30:02.223891 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:30:02.223891 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:30:02.224720 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:30:02.224720 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:30:02.224720 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:30:02.224720 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:30:02.224720 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:30:02.224720 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:30:02.224720 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:30:02.224720 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:30:02.224720 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:30:02.224720 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:30:02.224720 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 17:30:02.224720 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:30:02.227463 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 134/256] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jun 16 17:30:01 2022] GET /identity/v3/roles/reader => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:30:02.228759 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67a0055-5869-41b5-b77b-6d3d42c6c0a8 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:02.231464 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b637a97-1b69-4ab2-b890-2d5e6b8e6565 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:02.279425 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ee754e-e849-4131-a1b0-aa5afac6dd69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:02.280414 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ee754e-e849-4131-a1b0-aa5afac6dd69 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:02.280642 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ee754e-e849-4131-a1b0-aa5afac6dd69 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:02.280859 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ee754e-e849-4131-a1b0-aa5afac6dd69 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:02.294816 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ee754e-e849-4131-a1b0-aa5afac6dd69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:30:02.304270 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-d4ee754e-e849-4131-a1b0-aa5afac6dd69 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 17:30:02.304270 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:30:02.304270 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:30:02.304270 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:30:02.304270 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:30:02.304270 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:30:02.304270 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:30:02.304270 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:30:02.304270 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:30:02.304270 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:30:02.304270 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:30:02.304270 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:30:02.304270 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:30:02.304270 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:30:02.304270 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:30:02.304270 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:30:02.305229 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:30:02.305229 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:30:02.305229 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:30:02.305229 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:30:02.305229 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:30:02.305229 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:30:02.305229 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:30:02.305229 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:30:02.305229 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:30:02.305229 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:30:02.305229 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:30:02.305229 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:30:02.305229 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:30:02.305229 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:30:02.305229 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:30:02.305229 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:30:02.306007 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:30:02.306007 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:30:02.306007 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:30:02.306007 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:30:02.306007 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:30:02.306007 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:30:02.306007 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:30:02.306007 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:30:02.306007 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:30:02.306007 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:30:02.306007 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 17:30:02.306007 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:30:02.306007 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 123/257] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Jun 16 17:30:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:30:02.306674 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ee754e-e849-4131-a1b0-aa5afac6dd69 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:02.361958 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67a0055-5869-41b5-b77b-6d3d42c6c0a8 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:02.365200 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:02.412299 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c211c261-881d-43f3-b070-2b7cb1fcadf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:02.413493 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c211c261-881d-43f3-b070-2b7cb1fcadf5 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:02.413775 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c211c261-881d-43f3-b070-2b7cb1fcadf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:02.414050 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c211c261-881d-43f3-b070-2b7cb1fcadf5 None admin] PATH_INFO: `/v3/projects/d820cdcb6c0741e380190e6fe127e992` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:02.415936 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c211c261-881d-43f3-b070-2b7cb1fcadf5 None admin] RBAC: Authorizing `identity:get_project(project_id=d820cdcb6c0741e380190e6fe127e992)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:02.417945 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c211c261-881d-43f3-b070-2b7cb1fcadf5 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:02.420591 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 135/258] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jun 16 17:30:02 2022] GET /identity/v3/projects/d820cdcb6c0741e380190e6fe127e992 => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:02.421297 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c211c261-881d-43f3-b070-2b7cb1fcadf5 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:02.431827 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ee754e-e849-4131-a1b0-aa5afac6dd69 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:02.452222 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:02.477036 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c732028-c58e-4bef-b440-08868f2a8806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:02.477942 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c732028-c58e-4bef-b440-08868f2a8806 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:02.478328 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c732028-c58e-4bef-b440-08868f2a8806 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:02.478705 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c732028-c58e-4bef-b440-08868f2a8806 None admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:02.479674 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c732028-c58e-4bef-b440-08868f2a8806 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:02.480082 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c732028-c58e-4bef-b440-08868f2a8806 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:02.481725 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c732028-c58e-4bef-b440-08868f2a8806 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:02.492332 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 124/259] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Jun 16 17:30:02 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:02.493490 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c732028-c58e-4bef-b440-08868f2a8806 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:02.559261 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c211c261-881d-43f3-b070-2b7cb1fcadf5 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:02.601419 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce12447-fc1a-4f89-a41e-1d69e5032915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:02.602203 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce12447-fc1a-4f89-a41e-1d69e5032915 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:02.602488 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce12447-fc1a-4f89-a41e-1d69e5032915 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:02.602763 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce12447-fc1a-4f89-a41e-1d69e5032915 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:02.603511 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce12447-fc1a-4f89-a41e-1d69e5032915 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:02.603814 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce12447-fc1a-4f89-a41e-1d69e5032915 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:02.605461 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce12447-fc1a-4f89-a41e-1d69e5032915 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:02.611182 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c732028-c58e-4bef-b440-08868f2a8806 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:02.615410 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 136/260] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jun 16 17:30:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:02.616274 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce12447-fc1a-4f89-a41e-1d69e5032915 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:02.652294 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45041b7-11a1-4d0a-835d-0745c481a177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:02.653325 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45041b7-11a1-4d0a-835d-0745c481a177 None admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:02.653612 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45041b7-11a1-4d0a-835d-0745c481a177 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:02.653892 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45041b7-11a1-4d0a-835d-0745c481a177 None admin] PATH_INFO: `/v3/projects/d820cdcb6c0741e380190e6fe127e992/users/f49bf79057404437b9ab776a195d6339/roles/0be2075946f54c8485f1c275f746478a` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:02.655919 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45041b7-11a1-4d0a-835d-0745c481a177 None admin] RBAC: Authorizing `identity:create_grant(project_id=d820cdcb6c0741e380190e6fe127e992, user_id=f49bf79057404437b9ab776a195d6339, role_id=0be2075946f54c8485f1c275f746478a)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:02.658188 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45041b7-11a1-4d0a-835d-0745c481a177 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:02.659490 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:02.672644 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 125/261] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Jun 16 17:30:02 2022] PUT /identity/v3/projects/d820cdcb6c0741e380190e6fe127e992/users/f49bf79057404437b9ab776a195d6339/roles/0be2075946f54c8485f1c275f746478a => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:30:02.673718 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45041b7-11a1-4d0a-835d-0745c481a177 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:02.737866 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce12447-fc1a-4f89-a41e-1d69e5032915 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:02.755574 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:02.780110 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3b6415-56cc-42ca-85c4-4bac8d06db7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:02.781022 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3b6415-56cc-42ca-85c4-4bac8d06db7b None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:02.781304 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3b6415-56cc-42ca-85c4-4bac8d06db7b None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:02.782259 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3b6415-56cc-42ca-85c4-4bac8d06db7b None admin] PATH_INFO: `/v3/users/8b1a9c6575ea431fb712662972df2fb3` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:02.797977 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3b6415-56cc-42ca-85c4-4bac8d06db7b None admin] RBAC: Authorizing `identity:get_user(user_id=8b1a9c6575ea431fb712662972df2fb3)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:02.799458 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3b6415-56cc-42ca-85c4-4bac8d06db7b None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:02.801476 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 137/262] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jun 16 17:30:02 2022] GET /identity/v3/users/8b1a9c6575ea431fb712662972df2fb3 => generated 297 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:02.807096 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3b6415-56cc-42ca-85c4-4bac8d06db7b None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:02.871292 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45041b7-11a1-4d0a-835d-0745c481a177 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:02.913617 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:02.922646 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3b6415-56cc-42ca-85c4-4bac8d06db7b None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:02.947778 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88330fe9-3e0d-4165-b063-4a320f40a3b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:02.948707 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88330fe9-3e0d-4165-b063-4a320f40a3b2 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:02.948990 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88330fe9-3e0d-4165-b063-4a320f40a3b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:02.949267 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88330fe9-3e0d-4165-b063-4a320f40a3b2 None admin] PATH_INFO: `/v3/users/bc760a5800b649989e800249b203636f` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:02.954230 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88330fe9-3e0d-4165-b063-4a320f40a3b2 None admin] RBAC: Authorizing `identity:get_user(user_id=bc760a5800b649989e800249b203636f)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:02.955885 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88330fe9-3e0d-4165-b063-4a320f40a3b2 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:02.957686 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 126/263] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jun 16 17:30:02 2022] GET /identity/v3/users/bc760a5800b649989e800249b203636f => generated 283 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:02.964332 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88330fe9-3e0d-4165-b063-4a320f40a3b2 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:03.000043 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8091638d-976a-4b8a-bc9a-19c601108d51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:03.001140 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8091638d-976a-4b8a-bc9a-19c601108d51 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:03.001415 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8091638d-976a-4b8a-bc9a-19c601108d51 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:03.001692 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8091638d-976a-4b8a-bc9a-19c601108d51 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:03.086378 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8091638d-976a-4b8a-bc9a-19c601108d51 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:03.087866 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8091638d-976a-4b8a-bc9a-19c601108d51 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:03.103284 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88330fe9-3e0d-4165-b063-4a320f40a3b2 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:03.114225 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:03.142722 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119cdbe1-4595-4f4b-b2dc-703ee7fba7f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:03.143685 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119cdbe1-4595-4f4b-b2dc-703ee7fba7f4 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:03.144183 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119cdbe1-4595-4f4b-b2dc-703ee7fba7f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:03.144730 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119cdbe1-4595-4f4b-b2dc-703ee7fba7f4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:03.192167 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:03.208510 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 138/264] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jun 16 17:30:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:30:03.210850 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8091638d-976a-4b8a-bc9a-19c601108d51 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:03.233181 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119cdbe1-4595-4f4b-b2dc-703ee7fba7f4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:03.235411 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119cdbe1-4595-4f4b-b2dc-703ee7fba7f4 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:03.320528 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8091638d-976a-4b8a-bc9a-19c601108d51 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:03.350296 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 127/265] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jun 16 17:30:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:30:03.356070 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119cdbe1-4595-4f4b-b2dc-703ee7fba7f4 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:03.359860 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4132e3-9afd-45b7-8715-d08032739cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:03.360750 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4132e3-9afd-45b7-8715-d08032739cd9 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:03.361031 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4132e3-9afd-45b7-8715-d08032739cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:03.361308 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4132e3-9afd-45b7-8715-d08032739cd9 None admin] PATH_INFO: `/v3/projects/d820cdcb6c0741e380190e6fe127e992` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:03.362509 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4132e3-9afd-45b7-8715-d08032739cd9 None admin] RBAC: Authorizing `identity:get_project(project_id=d820cdcb6c0741e380190e6fe127e992)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:03.363963 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4132e3-9afd-45b7-8715-d08032739cd9 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:03.366233 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 139/266] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jun 16 17:30:03 2022] GET /identity/v3/projects/d820cdcb6c0741e380190e6fe127e992 => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:30:03.371132 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:03.373752 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4132e3-9afd-45b7-8715-d08032739cd9 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:03.439989 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:03.478867 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119cdbe1-4595-4f4b-b2dc-703ee7fba7f4 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:03.516529 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bd8989-9279-4132-a886-46d96d7251f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:03.517155 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4132e3-9afd-45b7-8715-d08032739cd9 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:03.517706 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bd8989-9279-4132-a886-46d96d7251f9 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:03.518090 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bd8989-9279-4132-a886-46d96d7251f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:03.520422 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bd8989-9279-4132-a886-46d96d7251f9 None admin] PATH_INFO: `/v3/projects/d820cdcb6c0741e380190e6fe127e992` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:03.525193 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bd8989-9279-4132-a886-46d96d7251f9 None admin] RBAC: Authorizing `identity:get_project(project_id=d820cdcb6c0741e380190e6fe127e992)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:03.526934 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bd8989-9279-4132-a886-46d96d7251f9 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:03.529934 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 128/267] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jun 16 17:30:03 2022] GET /identity/v3/projects/d820cdcb6c0741e380190e6fe127e992 => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:30:03.536222 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bd8989-9279-4132-a886-46d96d7251f9 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:03.593073 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e23fc57-0a89-455a-aa10-cdaac1e63380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:03.593747 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e23fc57-0a89-455a-aa10-cdaac1e63380 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:03.594097 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e23fc57-0a89-455a-aa10-cdaac1e63380 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:03.594376 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e23fc57-0a89-455a-aa10-cdaac1e63380 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:03.595540 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e23fc57-0a89-455a-aa10-cdaac1e63380 None admin] RBAC: Adding query filter params (role.id=9ed5382e55d649faa9195c4a9eb6fa57, scope.project.id=d820cdcb6c0741e380190e6fe127e992, user.id=8b1a9c6575ea431fb712662972df2fb3) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:03.595665 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e23fc57-0a89-455a-aa10-cdaac1e63380 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:03.597393 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e23fc57-0a89-455a-aa10-cdaac1e63380 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:03.603360 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 140/268] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jun 16 17:30:03 2022] GET /identity/v3/role_assignments?user.id=8b1a9c6575ea431fb712662972df2fb3&scope.project.id=d820cdcb6c0741e380190e6fe127e992&role.id=9ed5382e55d649faa9195c4a9eb6fa57 => generated 259 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:30:03.663982 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:03.691668 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bd8989-9279-4132-a886-46d96d7251f9 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:03.750860 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d9c928-9af0-4a7c-961f-1ef23f8ffc0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:03.751627 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d9c928-9af0-4a7c-961f-1ef23f8ffc0f None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:03.751916 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d9c928-9af0-4a7c-961f-1ef23f8ffc0f None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:03.752189 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d9c928-9af0-4a7c-961f-1ef23f8ffc0f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:03.753067 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d9c928-9af0-4a7c-961f-1ef23f8ffc0f None admin] RBAC: Adding query filter params (role.id=243fd0a11d94477986deb1dcfb3f31dd, scope.project.id=d820cdcb6c0741e380190e6fe127e992, user.id=bc760a5800b649989e800249b203636f) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:03.753379 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d9c928-9af0-4a7c-961f-1ef23f8ffc0f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:03.754857 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d9c928-9af0-4a7c-961f-1ef23f8ffc0f None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:03.765671 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 129/269] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jun 16 17:30:03 2022] GET /identity/v3/role_assignments?user.id=bc760a5800b649989e800249b203636f&scope.project.id=d820cdcb6c0741e380190e6fe127e992&role.id=243fd0a11d94477986deb1dcfb3f31dd => generated 608 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:30:04.233637 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca478d97-df7b-4cf3-9af8-113a2bad0f63 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:04.234144 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca478d97-df7b-4cf3-9af8-113a2bad0f63 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:04.234217 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca478d97-df7b-4cf3-9af8-113a2bad0f63 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:04.236514 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 141/270] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:30:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:30:04.241684 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5691db-1c27-4daa-ab9f-f6fb14d1cd16 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:04.242130 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5691db-1c27-4daa-ab9f-f6fb14d1cd16 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:04.242318 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5691db-1c27-4daa-ab9f-f6fb14d1cd16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:04.261653 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-8e5691db-1c27-4daa-ab9f-f6fb14d1cd16 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:30:04.262863 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-8e5691db-1c27-4daa-ab9f-f6fb14d1cd16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:30:04.349828 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 130/271] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:30:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:30:04.374548 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca478d97-df7b-4cf3-9af8-113a2bad0f63 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:04.494755 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca478d97-df7b-4cf3-9af8-113a2bad0f63 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:04.511360 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d88d00a-52db-4f66-a403-dd35c56394b4 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:04.511805 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d88d00a-52db-4f66-a403-dd35c56394b4 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:04.512218 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d88d00a-52db-4f66-a403-dd35c56394b4 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:04.514825 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 131/272] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:30:04 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:30:04.521456 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd050c7-6d52-4d20-a1cc-37f377b8febe None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:04.522055 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd050c7-6d52-4d20-a1cc-37f377b8febe None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:04.522523 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:04.523019 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd050c7-6d52-4d20-a1cc-37f377b8febe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:04.535019 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c2530a-b2c5-41ea-8429-1452dcdde119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:04.535823 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c2530a-b2c5-41ea-8429-1452dcdde119 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:04.536104 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c2530a-b2c5-41ea-8429-1452dcdde119 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:04.536409 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c2530a-b2c5-41ea-8429-1452dcdde119 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:04.544809 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-ccd050c7-6d52-4d20-a1cc-37f377b8febe None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:30:04.546296 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-ccd050c7-6d52-4d20-a1cc-37f377b8febe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:30:04.547711 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c2530a-b2c5-41ea-8429-1452dcdde119 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:30:04.555822 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-81c2530a-b2c5-41ea-8429-1452dcdde119 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:30:04.555822 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:30:04.555822 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:30:04.555822 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:30:04.555822 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:30:04.555822 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:30:04.555822 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:30:04.555822 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:30:04.555822 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:30:04.555822 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:30:04.555822 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:30:04.555822 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:30:04.555822 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:30:04.555822 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:30:04.555822 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:30:04.555822 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:30:04.556809 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:30:04.556809 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:30:04.556809 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:30:04.556809 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:30:04.556809 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:30:04.556809 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:30:04.556809 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:30:04.556809 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:30:04.556809 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:30:04.556809 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:30:04.556809 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:30:04.556809 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:30:04.556809 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:30:04.556809 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:30:04.556809 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:30:04.556809 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:30:04.557600 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:30:04.557600 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:30:04.557600 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:30:04.557600 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:30:04.557600 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:30:04.557600 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:30:04.557600 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:30:04.557600 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:30:04.557600 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:30:04.557600 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:30:04.557600 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:30:04.557600 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:30:04.557600 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 142/273] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jun 16 17:30:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:30:04.565645 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c2530a-b2c5-41ea-8429-1452dcdde119 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:04.626946 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:04.640402 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 132/274] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:30:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:30:04.659792 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd050c7-6d52-4d20-a1cc-37f377b8febe None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:04.702084 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c2530a-b2c5-41ea-8429-1452dcdde119 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:04.740204 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aa2fed-a698-4a9b-b62c-5f1228fbafb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:04.741385 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aa2fed-a698-4a9b-b62c-5f1228fbafb0 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:04.741862 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aa2fed-a698-4a9b-b62c-5f1228fbafb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:04.742377 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aa2fed-a698-4a9b-b62c-5f1228fbafb0 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:04.743306 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1aa2fed-a698-4a9b-b62c-5f1228fbafb0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:04.743802 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1aa2fed-a698-4a9b-b62c-5f1228fbafb0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:04.745553 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1aa2fed-a698-4a9b-b62c-5f1228fbafb0 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:04.754773 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 143/275] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Jun 16 17:30:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:04.761657 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aa2fed-a698-4a9b-b62c-5f1228fbafb0 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:04.792775 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd050c7-6d52-4d20-a1cc-37f377b8febe None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:04.830440 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:04.831716 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbfa2b6-7d93-4685-89bc-2b6c9b409a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:04.832864 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbfa2b6-7d93-4685-89bc-2b6c9b409a70 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:04.833348 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbfa2b6-7d93-4685-89bc-2b6c9b409a70 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:04.833815 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbfa2b6-7d93-4685-89bc-2b6c9b409a70 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:04.835189 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbfa2b6-7d93-4685-89bc-2b6c9b409a70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:04.836973 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbfa2b6-7d93-4685-89bc-2b6c9b409a70 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:04.839148 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 133/276] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jun 16 17:30:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:30:04.848403 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbfa2b6-7d93-4685-89bc-2b6c9b409a70 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:04.909210 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aa2fed-a698-4a9b-b62c-5f1228fbafb0 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:04.935849 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:04.956117 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96e0a82-4b8f-422b-afd8-5f36bdc26f55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:04.957043 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96e0a82-4b8f-422b-afd8-5f36bdc26f55 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:04.957326 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96e0a82-4b8f-422b-afd8-5f36bdc26f55 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:04.957601 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96e0a82-4b8f-422b-afd8-5f36bdc26f55 None admin] PATH_INFO: `/v3/users/497291fbb41246fb8c2150713937beb6` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:04.972787 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96e0a82-4b8f-422b-afd8-5f36bdc26f55 None admin] RBAC: Authorizing `identity:get_user(user_id=497291fbb41246fb8c2150713937beb6)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:04.974264 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96e0a82-4b8f-422b-afd8-5f36bdc26f55 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:04.976089 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 144/277] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jun 16 17:30:04 2022] GET /identity/v3/users/497291fbb41246fb8c2150713937beb6 => generated 293 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:04.976870 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbfa2b6-7d93-4685-89bc-2b6c9b409a70 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:04.982168 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96e0a82-4b8f-422b-afd8-5f36bdc26f55 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:05.036686 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3de71ea-54a8-4366-aac8-210e42ee8b20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:05.037389 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3de71ea-54a8-4366-aac8-210e42ee8b20 None admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:05.037682 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3de71ea-54a8-4366-aac8-210e42ee8b20 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:05.038065 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3de71ea-54a8-4366-aac8-210e42ee8b20 None admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:05.039053 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3de71ea-54a8-4366-aac8-210e42ee8b20 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:05.041514 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3de71ea-54a8-4366-aac8-210e42ee8b20 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:05.070265 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 134/278] 10.222.0.42 () {60 vars in 1132 bytes} [Thu Jun 16 17:30:04 2022] POST /identity/v3/users => generated 293 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:30:05.098179 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96e0a82-4b8f-422b-afd8-5f36bdc26f55 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:05.101692 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:05.146147 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f15967-c30f-4907-b63f-2e7ef7bf39ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:05.146819 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f15967-c30f-4907-b63f-2e7ef7bf39ff None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:05.147132 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f15967-c30f-4907-b63f-2e7ef7bf39ff None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:05.147420 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f15967-c30f-4907-b63f-2e7ef7bf39ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:05.148262 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f15967-c30f-4907-b63f-2e7ef7bf39ff None admin] RBAC: Adding query filter params (role.id=0be2075946f54c8485f1c275f746478a, user.id=497291fbb41246fb8c2150713937beb6, scope.system=all) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:05.148606 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f15967-c30f-4907-b63f-2e7ef7bf39ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:05.150261 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f15967-c30f-4907-b63f-2e7ef7bf39ff None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:05.161267 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 145/279] 10.222.0.42 () {56 vars in 1308 bytes} [Thu Jun 16 17:30:04 2022] GET /identity/v3/role_assignments?user.id=497291fbb41246fb8c2150713937beb6&scope.system=all&role.id=0be2075946f54c8485f1c275f746478a => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:30:05.585867 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138c816a-2f1b-49da-a247-f18eb5d2e585 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:05.586380 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138c816a-2f1b-49da-a247-f18eb5d2e585 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:05.586453 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138c816a-2f1b-49da-a247-f18eb5d2e585 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:05.588096 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 135/280] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:30:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:30:05.593178 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6969ab5a-42f7-4017-9e88-a726f25cf4a2 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:05.593629 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6969ab5a-42f7-4017-9e88-a726f25cf4a2 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:05.593844 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6969ab5a-42f7-4017-9e88-a726f25cf4a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:05.613020 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-6969ab5a-42f7-4017-9e88-a726f25cf4a2 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:30:05.614216 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-6969ab5a-42f7-4017-9e88-a726f25cf4a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:30:05.712731 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 146/281] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:30:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:30:05.734256 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138c816a-2f1b-49da-a247-f18eb5d2e585 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:05.870155 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138c816a-2f1b-49da-a247-f18eb5d2e585 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:05.904765 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:05.921544 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6705d134-2fbd-4936-aaa5-dc4dbcb2b3f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:05.922860 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6705d134-2fbd-4936-aaa5-dc4dbcb2b3f1 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:05.923360 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6705d134-2fbd-4936-aaa5-dc4dbcb2b3f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:05.923822 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6705d134-2fbd-4936-aaa5-dc4dbcb2b3f1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:05.942421 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6705d134-2fbd-4936-aaa5-dc4dbcb2b3f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:30:05.950908 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-6705d134-2fbd-4936-aaa5-dc4dbcb2b3f1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:30:05.950908 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:30:05.950908 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:30:05.950908 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:30:05.950908 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:30:05.950908 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:30:05.950908 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:30:05.950908 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:30:05.950908 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:30:05.950908 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:30:05.950908 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:30:05.950908 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:30:05.950908 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:30:05.950908 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:30:05.950908 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:30:05.950908 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:30:05.951786 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:30:05.951786 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:30:05.951786 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:30:05.951786 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:30:05.951786 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:30:05.951786 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:30:05.951786 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:30:05.951786 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:30:05.951786 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:30:05.951786 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:30:05.951786 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:30:05.951786 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:30:05.951786 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:30:05.951786 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:30:05.951786 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:30:05.951786 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:30:05.952595 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:30:05.952595 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:30:05.952595 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:30:05.952595 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:30:05.952595 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:30:05.952595 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:30:05.952595 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:30:05.952595 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:30:05.952595 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:30:05.952595 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:30:05.952595 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:30:05.952595 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:30:05.955563 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 136/282] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jun 16 17:30:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:30:05.964670 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6969ab5a-42f7-4017-9e88-a726f25cf4a2 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:06.092599 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:06.097591 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6969ab5a-42f7-4017-9e88-a726f25cf4a2 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:06.138498 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee29517-8b41-4f0d-9fc3-65981a101d43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:06.139096 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee29517-8b41-4f0d-9fc3-65981a101d43 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:06.139375 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee29517-8b41-4f0d-9fc3-65981a101d43 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:06.139648 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee29517-8b41-4f0d-9fc3-65981a101d43 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:06.140413 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee29517-8b41-4f0d-9fc3-65981a101d43 None admin] RBAC: Adding query filter params (name=member) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:06.141242 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee29517-8b41-4f0d-9fc3-65981a101d43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:06.142842 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee29517-8b41-4f0d-9fc3-65981a101d43 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:06.150821 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f97c055-7500-4be8-b1f3-0441ed97a49f None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:06.151140 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f97c055-7500-4be8-b1f3-0441ed97a49f None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:06.151387 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f97c055-7500-4be8-b1f3-0441ed97a49f None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:06.152534 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 147/283] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Jun 16 17:30:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:06.155997 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 137/284] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:30:06 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:30:06.164470 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee29517-8b41-4f0d-9fc3-65981a101d43 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:06.176985 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea95a3c1-2520-4c29-ae27-b22cccebb5e5 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:06.177330 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea95a3c1-2520-4c29-ae27-b22cccebb5e5 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:06.177609 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea95a3c1-2520-4c29-ae27-b22cccebb5e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:06.201456 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-ea95a3c1-2520-4c29-ae27-b22cccebb5e5 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:30:06.202658 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-ea95a3c1-2520-4c29-ae27-b22cccebb5e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:30:06.342088 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 138/285] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:30:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 17:30:06.359427 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee29517-8b41-4f0d-9fc3-65981a101d43 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:06.365441 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea95a3c1-2520-4c29-ae27-b22cccebb5e5 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:06.404453 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26bdef1-5b1e-4e68-8c20-a751c6f89a13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:06.405606 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26bdef1-5b1e-4e68-8c20-a751c6f89a13 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:06.405892 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26bdef1-5b1e-4e68-8c20-a751c6f89a13 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:06.406170 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26bdef1-5b1e-4e68-8c20-a751c6f89a13 None admin] PATH_INFO: `/v3/users/f49bf79057404437b9ab776a195d6339` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:06.407993 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26bdef1-5b1e-4e68-8c20-a751c6f89a13 None admin] RBAC: Authorizing `identity:get_user(user_id=f49bf79057404437b9ab776a195d6339)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:06.409476 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26bdef1-5b1e-4e68-8c20-a751c6f89a13 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:06.412405 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 148/286] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jun 16 17:30:06 2022] GET /identity/v3/users/f49bf79057404437b9ab776a195d6339 => generated 297 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:06.421606 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26bdef1-5b1e-4e68-8c20-a751c6f89a13 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:06.495925 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:06.516827 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea95a3c1-2520-4c29-ae27-b22cccebb5e5 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:06.533144 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:06.555043 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572da73e-8023-4dd1-839b-49965a930f80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:06.555848 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572da73e-8023-4dd1-839b-49965a930f80 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:06.556130 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572da73e-8023-4dd1-839b-49965a930f80 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:06.556537 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572da73e-8023-4dd1-839b-49965a930f80 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:06.568930 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572da73e-8023-4dd1-839b-49965a930f80 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:30:06.576751 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-572da73e-8023-4dd1-839b-49965a930f80 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 17:30:06.576751 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:30:06.576751 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:30:06.576751 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:30:06.576751 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:30:06.576751 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:30:06.576751 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:30:06.576751 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:30:06.576751 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:30:06.576751 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:30:06.576751 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:30:06.576751 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:30:06.576751 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:30:06.576751 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:30:06.576751 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:30:06.576751 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:30:06.577593 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:30:06.577593 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:30:06.577593 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:30:06.577593 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:30:06.577593 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:30:06.577593 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:30:06.577593 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:30:06.577593 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:30:06.577593 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:30:06.577593 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:30:06.577593 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:30:06.577593 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:30:06.577593 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:30:06.577593 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:30:06.577593 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:30:06.577593 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:30:06.579172 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:30:06.579172 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:30:06.579172 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:30:06.579172 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:30:06.579172 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:30:06.579172 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:30:06.579172 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:30:06.579172 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:30:06.579172 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:30:06.579172 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:30:06.579172 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 17:30:06.579172 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:30:06.579172 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 139/287] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jun 16 17:30:06 2022] GET /identity/v3/roles/reader => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:30:06.585316 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572da73e-8023-4dd1-839b-49965a930f80 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:06.690496 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26bdef1-5b1e-4e68-8c20-a751c6f89a13 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:06.725749 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db27c101-98c0-4709-96ef-9a75fb805c7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:06.726435 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db27c101-98c0-4709-96ef-9a75fb805c7e None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:06.726718 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db27c101-98c0-4709-96ef-9a75fb805c7e None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:06.726995 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db27c101-98c0-4709-96ef-9a75fb805c7e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:06.728099 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572da73e-8023-4dd1-839b-49965a930f80 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:06.795606 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daf2d06-7b78-4910-a340-4cda07c38031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:06.796306 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daf2d06-7b78-4910-a340-4cda07c38031 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:06.796723 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daf2d06-7b78-4910-a340-4cda07c38031 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:06.797123 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daf2d06-7b78-4910-a340-4cda07c38031 None admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:06.799104 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1daf2d06-7b78-4910-a340-4cda07c38031 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:06.799424 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1daf2d06-7b78-4910-a340-4cda07c38031 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:06.799942 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db27c101-98c0-4709-96ef-9a75fb805c7e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:06.800972 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1daf2d06-7b78-4910-a340-4cda07c38031 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:06.801511 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db27c101-98c0-4709-96ef-9a75fb805c7e None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:06.819851 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 140/288] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Jun 16 17:30:06 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:30:06.825309 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daf2d06-7b78-4910-a340-4cda07c38031 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:06.851781 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:06.880750 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:06.943505 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daf2d06-7b78-4910-a340-4cda07c38031 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:06.983446 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12de442-7051-43ec-83eb-a948251af036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:06.984332 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12de442-7051-43ec-83eb-a948251af036 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:06.985337 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12de442-7051-43ec-83eb-a948251af036 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:06.985652 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12de442-7051-43ec-83eb-a948251af036 None admin] PATH_INFO: `/v3/users/8b1a9c6575ea431fb712662972df2fb3` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:06.987049 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12de442-7051-43ec-83eb-a948251af036 None admin] RBAC: Authorizing `identity:get_user(user_id=8b1a9c6575ea431fb712662972df2fb3)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:06.988735 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12de442-7051-43ec-83eb-a948251af036 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:06.990825 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 141/289] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jun 16 17:30:06 2022] GET /identity/v3/users/8b1a9c6575ea431fb712662972df2fb3 => generated 297 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:30:06.996162 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12de442-7051-43ec-83eb-a948251af036 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:07.043658 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 149/290] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jun 16 17:30:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 622 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:30:07.055242 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db27c101-98c0-4709-96ef-9a75fb805c7e None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:07.120090 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12de442-7051-43ec-83eb-a948251af036 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:07.122983 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:07.164841 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e2bbde-0d9e-43ef-9726-0d416666fa80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:07.165721 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e2bbde-0d9e-43ef-9726-0d416666fa80 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:07.166007 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e2bbde-0d9e-43ef-9726-0d416666fa80 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:07.166281 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e2bbde-0d9e-43ef-9726-0d416666fa80 None admin] PATH_INFO: `/v3/projects/d820cdcb6c0741e380190e6fe127e992` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:07.167492 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e2bbde-0d9e-43ef-9726-0d416666fa80 None admin] RBAC: Authorizing `identity:get_project(project_id=d820cdcb6c0741e380190e6fe127e992)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:07.168991 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e2bbde-0d9e-43ef-9726-0d416666fa80 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:07.171488 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 142/291] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jun 16 17:30:06 2022] GET /identity/v3/projects/d820cdcb6c0741e380190e6fe127e992 => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:07.178833 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e2bbde-0d9e-43ef-9726-0d416666fa80 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:07.238245 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db27c101-98c0-4709-96ef-9a75fb805c7e None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:07.281374 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:07.297502 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9203295-023a-40b1-a68c-e3f938fab779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:07.298127 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e2bbde-0d9e-43ef-9726-0d416666fa80 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:07.298407 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9203295-023a-40b1-a68c-e3f938fab779 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:07.298665 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9203295-023a-40b1-a68c-e3f938fab779 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:07.298946 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9203295-023a-40b1-a68c-e3f938fab779 None admin] PATH_INFO: `/v3/projects/d820cdcb6c0741e380190e6fe127e992` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:07.305167 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9203295-023a-40b1-a68c-e3f938fab779 None admin] RBAC: Authorizing `identity:get_project(project_id=d820cdcb6c0741e380190e6fe127e992)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:07.306666 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9203295-023a-40b1-a68c-e3f938fab779 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:07.308798 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 150/292] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jun 16 17:30:07 2022] GET /identity/v3/projects/d820cdcb6c0741e380190e6fe127e992 => generated 302 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:30:07.315232 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9203295-023a-40b1-a68c-e3f938fab779 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:07.339392 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2690146a-95e0-460b-a2cd-43c7b34ea1fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:07.340773 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2690146a-95e0-460b-a2cd-43c7b34ea1fa None admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:07.341265 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2690146a-95e0-460b-a2cd-43c7b34ea1fa None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:07.341738 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2690146a-95e0-460b-a2cd-43c7b34ea1fa None admin] PATH_INFO: `/v3/projects/d820cdcb6c0741e380190e6fe127e992/users/8b1a9c6575ea431fb712662972df2fb3/roles/9ed5382e55d649faa9195c4a9eb6fa57` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:07.343982 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2690146a-95e0-460b-a2cd-43c7b34ea1fa None admin] RBAC: Authorizing `identity:create_grant(project_id=d820cdcb6c0741e380190e6fe127e992, user_id=8b1a9c6575ea431fb712662972df2fb3, role_id=9ed5382e55d649faa9195c4a9eb6fa57)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:07.346327 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2690146a-95e0-460b-a2cd-43c7b34ea1fa None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:07.367574 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 143/293] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Jun 16 17:30:07 2022] PUT /identity/v3/projects/d820cdcb6c0741e380190e6fe127e992/users/8b1a9c6575ea431fb712662972df2fb3/roles/9ed5382e55d649faa9195c4a9eb6fa57 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:30:07.477026 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9203295-023a-40b1-a68c-e3f938fab779 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:07.533863 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae419e6-0e3c-4289-ab8b-1738bf9c73d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:07.534912 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae419e6-0e3c-4289-ab8b-1738bf9c73d0 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:07.535417 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae419e6-0e3c-4289-ab8b-1738bf9c73d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:07.535891 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae419e6-0e3c-4289-ab8b-1738bf9c73d0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:07.536971 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae419e6-0e3c-4289-ab8b-1738bf9c73d0 None admin] RBAC: Adding query filter params (role.id=0be2075946f54c8485f1c275f746478a, scope.project.id=d820cdcb6c0741e380190e6fe127e992, user.id=f49bf79057404437b9ab776a195d6339) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:07.537488 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae419e6-0e3c-4289-ab8b-1738bf9c73d0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:07.539156 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae419e6-0e3c-4289-ab8b-1738bf9c73d0 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:07.550183 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 151/294] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jun 16 17:30:07 2022] GET /identity/v3/role_assignments?user.id=f49bf79057404437b9ab776a195d6339&scope.project.id=d820cdcb6c0741e380190e6fe127e992&role.id=0be2075946f54c8485f1c275f746478a => generated 608 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:07.806072 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb10457-4550-46d2-946e-59e27c63d570 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:07.807116 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb10457-4550-46d2-946e-59e27c63d570 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:07.807405 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb10457-4550-46d2-946e-59e27c63d570 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:07.809104 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 144/295] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:30:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:30:07.815170 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f4d4d5-683b-4052-8e90-f69a216e590f None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:07.815693 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f4d4d5-683b-4052-8e90-f69a216e590f None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:07.815769 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f4d4d5-683b-4052-8e90-f69a216e590f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:07.838728 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-b7f4d4d5-683b-4052-8e90-f69a216e590f None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:30:07.840069 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-b7f4d4d5-683b-4052-8e90-f69a216e590f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:30:07.908234 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ab8263-c05a-4bef-825d-83c66da3b709 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:07.908575 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ab8263-c05a-4bef-825d-83c66da3b709 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:07.908855 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ab8263-c05a-4bef-825d-83c66da3b709 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:07.911199 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 145/296] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:30:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:30:07.916128 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404fcff1-9c92-42d2-b297-3e0e6e2e0cc1 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:07.916441 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404fcff1-9c92-42d2-b297-3e0e6e2e0cc1 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:07.916732 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404fcff1-9c92-42d2-b297-3e0e6e2e0cc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:07.925372 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:07.939436 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-404fcff1-9c92-42d2-b297-3e0e6e2e0cc1 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:30:07.941117 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-404fcff1-9c92-42d2-b297-3e0e6e2e0cc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:30:07.947057 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 152/297] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:30:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:30:07.971635 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:07.979288 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f4d4d5-683b-4052-8e90-f69a216e590f None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:08.031192 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 146/298] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:30:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 17:30:08.058384 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404fcff1-9c92-42d2-b297-3e0e6e2e0cc1 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:08.128862 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f4d4d5-683b-4052-8e90-f69a216e590f None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:08.171236 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d763cab1-599f-4b0f-b7ad-15f80bd28ff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:08.172660 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d763cab1-599f-4b0f-b7ad-15f80bd28ff5 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:08.173020 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d763cab1-599f-4b0f-b7ad-15f80bd28ff5 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:08.173225 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d763cab1-599f-4b0f-b7ad-15f80bd28ff5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:08.186293 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d763cab1-599f-4b0f-b7ad-15f80bd28ff5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:30:08.194501 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-d763cab1-599f-4b0f-b7ad-15f80bd28ff5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 17:30:08.194501 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:30:08.194501 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:30:08.194501 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:30:08.194501 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:30:08.194501 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:30:08.194501 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:30:08.194501 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:30:08.194501 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:30:08.194501 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:30:08.194501 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:30:08.194501 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:30:08.194501 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:30:08.194501 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:30:08.194501 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:30:08.194501 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:30:08.195441 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:30:08.195441 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:30:08.195441 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:30:08.195441 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:30:08.195441 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:30:08.195441 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:30:08.195441 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:30:08.195441 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:30:08.195441 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:30:08.195441 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:30:08.195441 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:30:08.195441 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:30:08.195441 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:30:08.195441 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:30:08.195441 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:30:08.195441 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:30:08.196236 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:30:08.196236 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:30:08.196236 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:30:08.196236 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:30:08.196236 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:30:08.196236 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:30:08.196236 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:30:08.196236 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:30:08.196236 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:30:08.196236 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:30:08.196236 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 17:30:08.196236 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:30:08.197637 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404fcff1-9c92-42d2-b297-3e0e6e2e0cc1 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:08.197710 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 153/299] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jun 16 17:30:07 2022] GET /identity/v3/roles/reader => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:30:08.197710 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cd496b-9ddc-4c64-83b7-619d3e707973 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:08.197710 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cd496b-9ddc-4c64-83b7-619d3e707973 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:08.197710 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cd496b-9ddc-4c64-83b7-619d3e707973 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:08.199018 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 154/300] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:30:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:30:08.203656 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cd496b-9ddc-4c64-83b7-619d3e707973 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:08.240059 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5398f9a4-4db1-4bbb-83a3-aaaabc06033b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:08.240922 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5398f9a4-4db1-4bbb-83a3-aaaabc06033b None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:08.241198 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5398f9a4-4db1-4bbb-83a3-aaaabc06033b None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:08.241497 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5398f9a4-4db1-4bbb-83a3-aaaabc06033b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:08.253304 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5398f9a4-4db1-4bbb-83a3-aaaabc06033b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:30:08.257941 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:08.262372 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-5398f9a4-4db1-4bbb-83a3-aaaabc06033b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:30:08.262372 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:30:08.262372 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:30:08.262372 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:30:08.262372 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:30:08.262372 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:30:08.262372 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:30:08.262372 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:30:08.262372 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:30:08.262372 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:30:08.262372 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:30:08.262372 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:30:08.262372 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:30:08.262372 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:30:08.262372 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:30:08.262372 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:30:08.263267 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:30:08.263267 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:30:08.263267 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:30:08.263267 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:30:08.263267 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:30:08.263267 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:30:08.263267 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:30:08.263267 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:30:08.263267 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:30:08.263267 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:30:08.263267 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:30:08.263267 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:30:08.263267 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:30:08.263267 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:30:08.263267 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:30:08.263267 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:30:08.264312 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:30:08.264312 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:30:08.264312 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:30:08.264312 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:30:08.264312 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:30:08.264312 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:30:08.264312 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:30:08.264312 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:30:08.264312 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:30:08.264312 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:30:08.264312 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:30:08.264312 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:30:08.264312 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 147/301] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jun 16 17:30:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:30:08.265620 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3697fc96-b899-4673-b88e-97a5c7c0e562 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:08.265620 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3697fc96-b899-4673-b88e-97a5c7c0e562 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:08.266543 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3697fc96-b899-4673-b88e-97a5c7c0e562 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:08.283810 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-3697fc96-b899-4673-b88e-97a5c7c0e562 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:30:08.285049 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-3697fc96-b899-4673-b88e-97a5c7c0e562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:30:08.302989 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:08.321220 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cd496b-9ddc-4c64-83b7-619d3e707973 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:08.358352 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17396be-1d0f-45c9-ac39-e8828f65fe60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:08.359024 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17396be-1d0f-45c9-ac39-e8828f65fe60 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:08.359305 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17396be-1d0f-45c9-ac39-e8828f65fe60 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:08.359704 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17396be-1d0f-45c9-ac39-e8828f65fe60 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:08.360659 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17396be-1d0f-45c9-ac39-e8828f65fe60 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:08.360971 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17396be-1d0f-45c9-ac39-e8828f65fe60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:08.368596 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 148/302] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:30:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:30:08.368766 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17396be-1d0f-45c9-ac39-e8828f65fe60 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:08.368851 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3697fc96-b899-4673-b88e-97a5c7c0e562 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:08.378845 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 155/303] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Jun 16 17:30:08 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:08.382749 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17396be-1d0f-45c9-ac39-e8828f65fe60 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:08.483780 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3697fc96-b899-4673-b88e-97a5c7c0e562 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:08.502935 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17396be-1d0f-45c9-ac39-e8828f65fe60 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:08.517712 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:08.523288 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d46d1a-39a4-4d8d-9e0c-3c157f08219a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:08.523976 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d46d1a-39a4-4d8d-9e0c-3c157f08219a None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:08.524260 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d46d1a-39a4-4d8d-9e0c-3c157f08219a None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:08.524571 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d46d1a-39a4-4d8d-9e0c-3c157f08219a None admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:08.525401 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d46d1a-39a4-4d8d-9e0c-3c157f08219a None admin] RBAC: Adding query filter params (name=member) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:08.525715 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d46d1a-39a4-4d8d-9e0c-3c157f08219a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:08.527118 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d46d1a-39a4-4d8d-9e0c-3c157f08219a None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:08.536605 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 149/304] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Jun 16 17:30:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:08.537544 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d46d1a-39a4-4d8d-9e0c-3c157f08219a None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:08.547675 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e067822b-37dd-4d35-9647-31e9b182457c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:08.548611 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e067822b-37dd-4d35-9647-31e9b182457c None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:08.548895 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e067822b-37dd-4d35-9647-31e9b182457c None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:08.549171 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e067822b-37dd-4d35-9647-31e9b182457c None admin] PATH_INFO: `/v3/users/f1a92b3cef3d4ddfa56b6c7476124fb4` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:08.555425 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:08.565804 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e067822b-37dd-4d35-9647-31e9b182457c None admin] RBAC: Authorizing `identity:get_user(user_id=f1a92b3cef3d4ddfa56b6c7476124fb4)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:08.567319 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e067822b-37dd-4d35-9647-31e9b182457c None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:08.573820 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 156/305] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jun 16 17:30:08 2022] GET /identity/v3/users/f1a92b3cef3d4ddfa56b6c7476124fb4 => generated 293 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:08.573820 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e067822b-37dd-4d35-9647-31e9b182457c None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:08.654821 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d46d1a-39a4-4d8d-9e0c-3c157f08219a None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:08.683662 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e067822b-37dd-4d35-9647-31e9b182457c None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:08.691790 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1bc125-9172-4171-85fe-4b23df471cf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:08.692620 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1bc125-9172-4171-85fe-4b23df471cf9 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:08.692908 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1bc125-9172-4171-85fe-4b23df471cf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:08.693185 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1bc125-9172-4171-85fe-4b23df471cf9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:08.694400 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1bc125-9172-4171-85fe-4b23df471cf9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:08.696932 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1bc125-9172-4171-85fe-4b23df471cf9 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:08.698754 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 150/306] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jun 16 17:30:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:08.699473 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1bc125-9172-4171-85fe-4b23df471cf9 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:08.727854 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0b7258-5b12-49f8-b376-6dcf76f7b2c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:08.728779 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0b7258-5b12-49f8-b376-6dcf76f7b2c1 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:08.729061 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0b7258-5b12-49f8-b376-6dcf76f7b2c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:08.729333 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0b7258-5b12-49f8-b376-6dcf76f7b2c1 None admin] PATH_INFO: `/v3/users/497291fbb41246fb8c2150713937beb6` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:08.730532 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0b7258-5b12-49f8-b376-6dcf76f7b2c1 None admin] RBAC: Authorizing `identity:get_user(user_id=497291fbb41246fb8c2150713937beb6)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:08.732054 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0b7258-5b12-49f8-b376-6dcf76f7b2c1 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:08.733772 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 157/307] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jun 16 17:30:08 2022] GET /identity/v3/users/497291fbb41246fb8c2150713937beb6 => generated 293 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:08.734336 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0b7258-5b12-49f8-b376-6dcf76f7b2c1 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:08.777170 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:08.795017 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:08.817305 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1bc125-9172-4171-85fe-4b23df471cf9 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:08.844402 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0b7258-5b12-49f8-b376-6dcf76f7b2c1 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:08.855955 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a97c367-52de-4ef1-865f-86675c6d626b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:08.856653 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a97c367-52de-4ef1-865f-86675c6d626b None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:08.856947 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a97c367-52de-4ef1-865f-86675c6d626b None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:08.857229 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a97c367-52de-4ef1-865f-86675c6d626b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:08.858062 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a97c367-52de-4ef1-865f-86675c6d626b None admin] RBAC: Adding query filter params (role.id=9ed5382e55d649faa9195c4a9eb6fa57, user.id=f1a92b3cef3d4ddfa56b6c7476124fb4, scope.system=all) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:08.858377 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a97c367-52de-4ef1-865f-86675c6d626b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:08.859919 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a97c367-52de-4ef1-865f-86675c6d626b None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:08.865486 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 151/308] 10.222.0.42 () {56 vars in 1308 bytes} [Thu Jun 16 17:30:08 2022] GET /identity/v3/role_assignments?user.id=f1a92b3cef3d4ddfa56b6c7476124fb4&scope.system=all&role.id=9ed5382e55d649faa9195c4a9eb6fa57 => generated 226 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:08.866192 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a97c367-52de-4ef1-865f-86675c6d626b None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:08.899957 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51866dc4-331e-42cd-b53a-f5ec0f12297b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:08.901438 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51866dc4-331e-42cd-b53a-f5ec0f12297b None admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:08.901963 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51866dc4-331e-42cd-b53a-f5ec0f12297b None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:08.902463 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51866dc4-331e-42cd-b53a-f5ec0f12297b None admin] PATH_INFO: `/v3/groups` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:08.903437 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51866dc4-331e-42cd-b53a-f5ec0f12297b None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:08.906077 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51866dc4-331e-42cd-b53a-f5ec0f12297b None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:08.959202 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 158/309] 10.222.0.42 () {60 vars in 1134 bytes} [Thu Jun 16 17:30:08 2022] POST /identity/v3/groups => generated 229 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:30:09.049391 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a97c367-52de-4ef1-865f-86675c6d626b None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:09.110172 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e41355-ed2d-489e-b9d3-7ba7fcb77e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:09.111296 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e41355-ed2d-489e-b9d3-7ba7fcb77e99 None admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:09.111723 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e41355-ed2d-489e-b9d3-7ba7fcb77e99 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:09.112141 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e41355-ed2d-489e-b9d3-7ba7fcb77e99 None admin] PATH_INFO: `/v3/system/users/497291fbb41246fb8c2150713937beb6/roles/0be2075946f54c8485f1c275f746478a` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:09.117341 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e41355-ed2d-489e-b9d3-7ba7fcb77e99 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=497291fbb41246fb8c2150713937beb6, role_id=0be2075946f54c8485f1c275f746478a)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:09.118992 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e41355-ed2d-489e-b9d3-7ba7fcb77e99 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:09.154676 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 152/310] 10.222.0.42 () {58 vars in 1334 bytes} [Thu Jun 16 17:30:08 2022] PUT /identity/v3/system/users/497291fbb41246fb8c2150713937beb6/roles/0be2075946f54c8485f1c275f746478a => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:30:09.865529 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5c342c-ff3b-410e-9a58-0b2722e10512 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:09.865529 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5c342c-ff3b-410e-9a58-0b2722e10512 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:09.865529 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5c342c-ff3b-410e-9a58-0b2722e10512 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:09.866212 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 159/311] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:30:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:30:09.872498 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e43dde-8721-4c26-81e3-c0948ec01b52 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:09.873337 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e43dde-8721-4c26-81e3-c0948ec01b52 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:09.873814 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e43dde-8721-4c26-81e3-c0948ec01b52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:09.898903 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-35e43dde-8721-4c26-81e3-c0948ec01b52 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:30:09.900452 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-35e43dde-8721-4c26-81e3-c0948ec01b52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:30:09.903675 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:09.985065 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 153/312] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:30:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:30:10.005522 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5c342c-ff3b-410e-9a58-0b2722e10512 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:10.108813 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:10.114485 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5c342c-ff3b-410e-9a58-0b2722e10512 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:10.153633 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde811e6-af08-4def-b67b-3d4de45b46ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:10.154345 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde811e6-af08-4def-b67b-3d4de45b46ca None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:10.154624 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde811e6-af08-4def-b67b-3d4de45b46ca None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:10.154898 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde811e6-af08-4def-b67b-3d4de45b46ca None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:10.165960 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde811e6-af08-4def-b67b-3d4de45b46ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:30:10.175634 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-cde811e6-af08-4def-b67b-3d4de45b46ca None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 17:30:10.175634 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:30:10.175634 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:30:10.175634 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:30:10.175634 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:30:10.175634 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:30:10.175634 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:30:10.175634 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:30:10.175634 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:30:10.175634 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:30:10.175634 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:30:10.175634 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:30:10.175634 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:30:10.175634 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:30:10.175634 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:30:10.175634 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:30:10.176634 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:30:10.176634 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:30:10.176634 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:30:10.176634 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:30:10.176634 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:30:10.176634 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:30:10.176634 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:30:10.176634 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:30:10.176634 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:30:10.176634 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:30:10.176634 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:30:10.176634 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:30:10.176634 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:30:10.176634 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:30:10.176634 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:30:10.176634 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:30:10.177460 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:30:10.177460 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:30:10.177460 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:30:10.177460 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:30:10.177460 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:30:10.177460 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:30:10.177460 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:30:10.177460 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:30:10.177460 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:30:10.177460 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:30:10.177460 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 17:30:10.177460 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:30:10.177460 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 160/313] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jun 16 17:30:10 2022] GET /identity/v3/roles/reader => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:30:10.184288 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e43dde-8721-4c26-81e3-c0948ec01b52 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:10.310653 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e43dde-8721-4c26-81e3-c0948ec01b52 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:10.348917 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4e68a2-3d57-4cf1-bd0f-9c6d2f87f0f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:10.349596 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4e68a2-3d57-4cf1-bd0f-9c6d2f87f0f4 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:10.349883 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4e68a2-3d57-4cf1-bd0f-9c6d2f87f0f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:10.350164 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4e68a2-3d57-4cf1-bd0f-9c6d2f87f0f4 None admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:10.350918 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4e68a2-3d57-4cf1-bd0f-9c6d2f87f0f4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:10.351234 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4e68a2-3d57-4cf1-bd0f-9c6d2f87f0f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:10.352646 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4e68a2-3d57-4cf1-bd0f-9c6d2f87f0f4 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:10.366376 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 154/314] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Jun 16 17:30:10 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:10.369455 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde811e6-af08-4def-b67b-3d4de45b46ca None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:10.475464 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde811e6-af08-4def-b67b-3d4de45b46ca None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:10.510176 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f77ff0-aaa1-4f5a-a05a-331c796993d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:10.511044 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f77ff0-aaa1-4f5a-a05a-331c796993d5 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:10.511322 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f77ff0-aaa1-4f5a-a05a-331c796993d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:10.511593 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f77ff0-aaa1-4f5a-a05a-331c796993d5 None admin] PATH_INFO: `/v3/users/8b1a9c6575ea431fb712662972df2fb3` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:10.512806 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f77ff0-aaa1-4f5a-a05a-331c796993d5 None admin] RBAC: Authorizing `identity:get_user(user_id=8b1a9c6575ea431fb712662972df2fb3)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:10.514505 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f77ff0-aaa1-4f5a-a05a-331c796993d5 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:10.516535 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 161/315] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jun 16 17:30:10 2022] GET /identity/v3/users/8b1a9c6575ea431fb712662972df2fb3 => generated 297 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:10.521868 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4e68a2-3d57-4cf1-bd0f-9c6d2f87f0f4 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:10.536728 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:10.631400 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4e68a2-3d57-4cf1-bd0f-9c6d2f87f0f4 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:10.666775 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a3ead7-f281-472f-ab6c-d9f1dd1fc7cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:10.667432 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a3ead7-f281-472f-ab6c-d9f1dd1fc7cd None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:10.667719 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a3ead7-f281-472f-ab6c-d9f1dd1fc7cd None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:10.667995 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a3ead7-f281-472f-ab6c-d9f1dd1fc7cd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:10.740137 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a3ead7-f281-472f-ab6c-d9f1dd1fc7cd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:10.741576 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a3ead7-f281-472f-ab6c-d9f1dd1fc7cd None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:10.743491 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:10.851696 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 155/316] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jun 16 17:30:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:30:10.853239 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f77ff0-aaa1-4f5a-a05a-331c796993d5 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:10.883956 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:10.966816 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f77ff0-aaa1-4f5a-a05a-331c796993d5 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:11.003624 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338695eb-d803-47ce-94dd-0ec145f065fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:11.004504 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338695eb-d803-47ce-94dd-0ec145f065fc None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:11.004793 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338695eb-d803-47ce-94dd-0ec145f065fc None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:11.005076 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338695eb-d803-47ce-94dd-0ec145f065fc None admin] PATH_INFO: `/v3/projects/d820cdcb6c0741e380190e6fe127e992` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:11.006318 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338695eb-d803-47ce-94dd-0ec145f065fc None admin] RBAC: Authorizing `identity:get_project(project_id=d820cdcb6c0741e380190e6fe127e992)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:11.007774 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338695eb-d803-47ce-94dd-0ec145f065fc None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:11.014841 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 162/317] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jun 16 17:30:10 2022] GET /identity/v3/projects/d820cdcb6c0741e380190e6fe127e992 => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:11.016905 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a3ead7-f281-472f-ab6c-d9f1dd1fc7cd None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:11.123379 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a3ead7-f281-472f-ab6c-d9f1dd1fc7cd None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:11.125639 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:11.162338 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8c657e-9bf6-4b87-8d95-494e4affab0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:11.163009 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8c657e-9bf6-4b87-8d95-494e4affab0a None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:11.163297 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8c657e-9bf6-4b87-8d95-494e4affab0a None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:11.163578 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8c657e-9bf6-4b87-8d95-494e4affab0a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:11.164453 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8c657e-9bf6-4b87-8d95-494e4affab0a None admin] RBAC: Adding query filter params (role.id=9ed5382e55d649faa9195c4a9eb6fa57, scope.project.id=d820cdcb6c0741e380190e6fe127e992, user.id=8b1a9c6575ea431fb712662972df2fb3) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:11.164775 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8c657e-9bf6-4b87-8d95-494e4affab0a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:11.166240 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8c657e-9bf6-4b87-8d95-494e4affab0a None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:11.180759 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 156/318] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jun 16 17:30:11 2022] GET /identity/v3/role_assignments?user.id=8b1a9c6575ea431fb712662972df2fb3&scope.project.id=d820cdcb6c0741e380190e6fe127e992&role.id=9ed5382e55d649faa9195c4a9eb6fa57 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:11.270905 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c35cf27-f56b-48fb-b890-4f232327a6e4 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:11.271564 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c35cf27-f56b-48fb-b890-4f232327a6e4 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:11.272061 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c35cf27-f56b-48fb-b890-4f232327a6e4 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:11.274178 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 163/319] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:30:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:30:11.280729 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac92124-c1d2-4454-9d1d-26db9e830a25 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:11.282037 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac92124-c1d2-4454-9d1d-26db9e830a25 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:11.282626 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac92124-c1d2-4454-9d1d-26db9e830a25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:11.302925 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-0ac92124-c1d2-4454-9d1d-26db9e830a25 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:30:11.305672 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-0ac92124-c1d2-4454-9d1d-26db9e830a25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:30:11.363040 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb33bec-530a-4843-8e4d-6165b7104fd0 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:11.363306 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb33bec-530a-4843-8e4d-6165b7104fd0 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:11.363604 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb33bec-530a-4843-8e4d-6165b7104fd0 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:11.365007 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 164/320] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:30:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:30:11.370453 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bc57b5-4aa5-4bc8-a910-39a53959e56f None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:11.370745 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bc57b5-4aa5-4bc8-a910-39a53959e56f None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:11.371027 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bc57b5-4aa5-4bc8-a910-39a53959e56f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:11.390078 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-98bc57b5-4aa5-4bc8-a910-39a53959e56f None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:30:11.392469 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 157/321] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:30:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:30:11.397961 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-98bc57b5-4aa5-4bc8-a910-39a53959e56f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:30:11.418736 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac92124-c1d2-4454-9d1d-26db9e830a25 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:11.494398 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 165/322] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:30:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:30:11.518362 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bc57b5-4aa5-4bc8-a910-39a53959e56f None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:11.544865 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:11.546144 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:11.553226 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac92124-c1d2-4454-9d1d-26db9e830a25 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:11.587507 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b3b6de-215f-420d-a21f-ee915ca8a956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:11.588298 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b3b6de-215f-420d-a21f-ee915ca8a956 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:11.588608 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b3b6de-215f-420d-a21f-ee915ca8a956 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:11.588880 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b3b6de-215f-420d-a21f-ee915ca8a956 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:11.598897 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b3b6de-215f-420d-a21f-ee915ca8a956 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:30:11.606541 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-77b3b6de-215f-420d-a21f-ee915ca8a956 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 17:30:11.606541 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:30:11.606541 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:30:11.606541 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:30:11.606541 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:30:11.606541 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:30:11.606541 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:30:11.606541 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:30:11.606541 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:30:11.606541 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:30:11.606541 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:30:11.606541 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:30:11.606541 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:30:11.606541 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:30:11.606541 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:30:11.606541 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:30:11.607720 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:30:11.607720 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:30:11.607720 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:30:11.607720 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:30:11.607720 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:30:11.607720 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:30:11.607720 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:30:11.607720 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:30:11.607720 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:30:11.607720 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:30:11.607720 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:30:11.607720 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:30:11.607720 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:30:11.607720 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:30:11.607720 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:30:11.607720 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:30:11.608600 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:30:11.608600 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:30:11.608600 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:30:11.608600 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:30:11.608600 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:30:11.608600 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:30:11.608600 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:30:11.608600 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:30:11.608600 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:30:11.608600 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:30:11.608600 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 17:30:11.608600 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:30:11.608600 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 158/323] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jun 16 17:30:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:30:11.609299 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7743649-aad9-45e8-8629-d3118b8cfdbd None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:11.609299 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7743649-aad9-45e8-8629-d3118b8cfdbd None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:11.610490 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7743649-aad9-45e8-8629-d3118b8cfdbd None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:11.612413 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 159/324] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:30:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:30:11.615024 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7743649-aad9-45e8-8629-d3118b8cfdbd None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:11.623430 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bc57b5-4aa5-4bc8-a910-39a53959e56f None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:11.658350 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1d7890-b2f5-480b-9769-b7143c7d7a8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:11.659147 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1d7890-b2f5-480b-9769-b7143c7d7a8d None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:11.659427 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1d7890-b2f5-480b-9769-b7143c7d7a8d None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:11.659703 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1d7890-b2f5-480b-9769-b7143c7d7a8d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:11.660989 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1d7890-b2f5-480b-9769-b7143c7d7a8d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:11.662423 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1d7890-b2f5-480b-9769-b7143c7d7a8d None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:11.664060 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 166/325] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jun 16 17:30:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:30:11.665307 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67069e7-26f9-422f-ba95-864cfdc9715b None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:11.665597 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67069e7-26f9-422f-ba95-864cfdc9715b None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:11.665875 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67069e7-26f9-422f-ba95-864cfdc9715b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:11.683044 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-e67069e7-26f9-422f-ba95-864cfdc9715b None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:30:11.684234 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-e67069e7-26f9-422f-ba95-864cfdc9715b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:30:11.724586 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7743649-aad9-45e8-8629-d3118b8cfdbd None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:11.752147 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 167/326] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:30:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:30:11.753080 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67069e7-26f9-422f-ba95-864cfdc9715b None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:11.760556 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25c1937-2c66-487f-8215-b06e59daf2c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:11.761530 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25c1937-2c66-487f-8215-b06e59daf2c8 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:11.762071 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25c1937-2c66-487f-8215-b06e59daf2c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:11.762537 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25c1937-2c66-487f-8215-b06e59daf2c8 None admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:11.763738 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25c1937-2c66-487f-8215-b06e59daf2c8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:11.764270 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25c1937-2c66-487f-8215-b06e59daf2c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:11.765936 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25c1937-2c66-487f-8215-b06e59daf2c8 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:11.775593 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 160/327] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Jun 16 17:30:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:30:11.776541 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25c1937-2c66-487f-8215-b06e59daf2c8 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:11.779346 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:11.798681 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:11.857865 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67069e7-26f9-422f-ba95-864cfdc9715b None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:11.882750 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25c1937-2c66-487f-8215-b06e59daf2c8 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:11.897022 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aa23da-7f5c-4844-a498-ab5047aac2f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:11.897704 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aa23da-7f5c-4844-a498-ab5047aac2f0 None admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:11.897989 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aa23da-7f5c-4844-a498-ab5047aac2f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:11.898267 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aa23da-7f5c-4844-a498-ab5047aac2f0 None admin] PATH_INFO: `/v3/groups` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:11.898958 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7aa23da-7f5c-4844-a498-ab5047aac2f0 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:11.900424 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7aa23da-7f5c-4844-a498-ab5047aac2f0 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:11.915044 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 168/328] 10.222.0.42 () {60 vars in 1134 bytes} [Thu Jun 16 17:30:11 2022] POST /identity/v3/groups => generated 226 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:30:11.915861 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aa23da-7f5c-4844-a498-ab5047aac2f0 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:11.930960 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b118bd-84fe-4cca-92c3-e1df178e3103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:11.931953 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b118bd-84fe-4cca-92c3-e1df178e3103 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:11.932307 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b118bd-84fe-4cca-92c3-e1df178e3103 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:11.932629 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b118bd-84fe-4cca-92c3-e1df178e3103 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:11.945536 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b118bd-84fe-4cca-92c3-e1df178e3103 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:30:11.954581 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-a9b118bd-84fe-4cca-92c3-e1df178e3103 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:30:11.954581 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:30:11.954581 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:30:11.954581 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:30:11.954581 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:30:11.954581 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:30:11.954581 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:30:11.954581 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:30:11.954581 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:30:11.954581 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:30:11.954581 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:30:11.954581 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:30:11.954581 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:30:11.954581 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:30:11.954581 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:30:11.954581 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:30:11.955549 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:30:11.955549 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:30:11.955549 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:30:11.955549 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:30:11.955549 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:30:11.955549 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:30:11.955549 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:30:11.955549 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:30:11.955549 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:30:11.955549 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:30:11.955549 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:30:11.955549 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:30:11.955549 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:30:11.955549 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:30:11.955549 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:30:11.955549 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:30:11.956362 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:30:11.956362 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:30:11.956362 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:30:11.956362 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:30:11.956362 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:30:11.956362 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:30:11.956362 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:30:11.956362 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:30:11.956362 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:30:11.956362 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:30:11.956362 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:30:11.956362 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:30:11.956362 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 161/329] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jun 16 17:30:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:30:11.962179 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b118bd-84fe-4cca-92c3-e1df178e3103 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:12.024598 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:12.049150 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aa23da-7f5c-4844-a498-ab5047aac2f0 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:12.061681 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:12.090222 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f53457-9b56-47bc-8942-4a67a2823a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:12.091403 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f53457-9b56-47bc-8942-4a67a2823a16 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:12.091692 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f53457-9b56-47bc-8942-4a67a2823a16 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:12.091973 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f53457-9b56-47bc-8942-4a67a2823a16 None admin] PATH_INFO: `/v3/users/f1a92b3cef3d4ddfa56b6c7476124fb4` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:12.092973 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b118bd-84fe-4cca-92c3-e1df178e3103 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:12.093264 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f53457-9b56-47bc-8942-4a67a2823a16 None admin] RBAC: Authorizing `identity:get_user(user_id=f1a92b3cef3d4ddfa56b6c7476124fb4)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:12.094912 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f53457-9b56-47bc-8942-4a67a2823a16 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:12.097453 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 169/330] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jun 16 17:30:11 2022] GET /identity/v3/users/f1a92b3cef3d4ddfa56b6c7476124fb4 => generated 293 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:12.102866 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f53457-9b56-47bc-8942-4a67a2823a16 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:12.135357 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f123e14-43ac-412c-8d27-332f83d1539e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:12.136055 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f123e14-43ac-412c-8d27-332f83d1539e None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:12.136535 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f123e14-43ac-412c-8d27-332f83d1539e None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:12.136654 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f123e14-43ac-412c-8d27-332f83d1539e None admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:12.137380 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f123e14-43ac-412c-8d27-332f83d1539e None admin] RBAC: Adding query filter params (name=member) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:12.137685 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f123e14-43ac-412c-8d27-332f83d1539e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:12.139328 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f123e14-43ac-412c-8d27-332f83d1539e None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:12.153534 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 162/331] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Jun 16 17:30:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:30:12.159342 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f123e14-43ac-412c-8d27-332f83d1539e None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:12.248584 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f53457-9b56-47bc-8942-4a67a2823a16 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:12.293508 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f123e14-43ac-412c-8d27-332f83d1539e None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:12.297922 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78889225-f2d8-46a4-8f56-f1e3be39e17b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:12.299111 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78889225-f2d8-46a4-8f56-f1e3be39e17b None admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:12.299660 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78889225-f2d8-46a4-8f56-f1e3be39e17b None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:12.300135 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78889225-f2d8-46a4-8f56-f1e3be39e17b None admin] PATH_INFO: `/v3/system/users/f1a92b3cef3d4ddfa56b6c7476124fb4/roles/9ed5382e55d649faa9195c4a9eb6fa57` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:12.301988 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78889225-f2d8-46a4-8f56-f1e3be39e17b None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=f1a92b3cef3d4ddfa56b6c7476124fb4, role_id=9ed5382e55d649faa9195c4a9eb6fa57)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:12.303610 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78889225-f2d8-46a4-8f56-f1e3be39e17b None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:12.313857 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 170/332] 10.222.0.42 () {58 vars in 1334 bytes} [Thu Jun 16 17:30:12 2022] PUT /identity/v3/system/users/f1a92b3cef3d4ddfa56b6c7476124fb4/roles/9ed5382e55d649faa9195c4a9eb6fa57 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:30:12.336995 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9495f38e-e974-4826-a1e3-a60bdf1acd93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:12.338470 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9495f38e-e974-4826-a1e3-a60bdf1acd93 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:12.338934 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9495f38e-e974-4826-a1e3-a60bdf1acd93 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:12.339396 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9495f38e-e974-4826-a1e3-a60bdf1acd93 None admin] PATH_INFO: `/v3/users/497291fbb41246fb8c2150713937beb6` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:12.341763 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9495f38e-e974-4826-a1e3-a60bdf1acd93 None admin] RBAC: Authorizing `identity:get_user(user_id=497291fbb41246fb8c2150713937beb6)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:12.345378 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9495f38e-e974-4826-a1e3-a60bdf1acd93 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:12.348224 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 163/333] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jun 16 17:30:12 2022] GET /identity/v3/users/497291fbb41246fb8c2150713937beb6 => generated 293 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:30:12.356007 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78889225-f2d8-46a4-8f56-f1e3be39e17b None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:12.375617 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:12.483177 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78889225-f2d8-46a4-8f56-f1e3be39e17b None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:12.524852 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaf34c9-a213-412a-b58d-27768aa8185d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:12.525512 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaf34c9-a213-412a-b58d-27768aa8185d None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:12.525796 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaf34c9-a213-412a-b58d-27768aa8185d None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:12.526072 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaf34c9-a213-412a-b58d-27768aa8185d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:12.526910 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaf34c9-a213-412a-b58d-27768aa8185d None admin] RBAC: Adding query filter params (role.id=0be2075946f54c8485f1c275f746478a, user.id=497291fbb41246fb8c2150713937beb6, scope.system=all) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:12.527223 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaf34c9-a213-412a-b58d-27768aa8185d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:12.529301 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaf34c9-a213-412a-b58d-27768aa8185d None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:12.542915 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 171/334] 10.222.0.42 () {56 vars in 1308 bytes} [Thu Jun 16 17:30:12 2022] GET /identity/v3/role_assignments?user.id=497291fbb41246fb8c2150713937beb6&scope.system=all&role.id=0be2075946f54c8485f1c275f746478a => generated 510 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:30:14.791478 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab9369f-52b7-4719-b1d2-5e09391592f5 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:14.791991 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab9369f-52b7-4719-b1d2-5e09391592f5 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:14.792095 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab9369f-52b7-4719-b1d2-5e09391592f5 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:14.794806 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 164/335] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:30:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:30:14.800652 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29076289-b2b6-42ed-8e84-38121c43250d None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:14.801113 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29076289-b2b6-42ed-8e84-38121c43250d None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:14.801206 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29076289-b2b6-42ed-8e84-38121c43250d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:14.830199 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-29076289-b2b6-42ed-8e84-38121c43250d None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:30:14.831440 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-29076289-b2b6-42ed-8e84-38121c43250d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:30:14.941025 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 172/336] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:30:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:30:14.974794 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab9369f-52b7-4719-b1d2-5e09391592f5 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:15.004647 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:15.008937 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aea941b-e15a-485a-a716-f639a38fb610 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:15.009396 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aea941b-e15a-485a-a716-f639a38fb610 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:15.009842 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aea941b-e15a-485a-a716-f639a38fb610 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:15.012448 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 173/337] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:30:15 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:30:15.019858 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830ef76d-f952-47cf-8701-7832178acbe3 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:15.020333 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830ef76d-f952-47cf-8701-7832178acbe3 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:15.020843 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830ef76d-f952-47cf-8701-7832178acbe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:15.045244 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:15.089988 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-830ef76d-f952-47cf-8701-7832178acbe3 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:30:15.091902 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-830ef76d-f952-47cf-8701-7832178acbe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:30:15.239601 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 174/338] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:30:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 17:30:15.241467 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edd8361-6e59-43ef-b391-99bafa795d15 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:15.241978 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edd8361-6e59-43ef-b391-99bafa795d15 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:15.242534 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edd8361-6e59-43ef-b391-99bafa795d15 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:15.248018 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 175/339] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:30:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:30:15.250076 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8d539d-5e97-4bce-a50a-25b4a9eae789 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:15.251394 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8d539d-5e97-4bce-a50a-25b4a9eae789 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:15.252107 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8d539d-5e97-4bce-a50a-25b4a9eae789 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:15.273297 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-ee8d539d-5e97-4bce-a50a-25b4a9eae789 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:30:15.274818 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-ee8d539d-5e97-4bce-a50a-25b4a9eae789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:30:15.320483 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab9369f-52b7-4719-b1d2-5e09391592f5 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:15.380619 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d468ab-249a-452f-b5d4-8f613781ea74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:15.381862 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d468ab-249a-452f-b5d4-8f613781ea74 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:15.382383 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d468ab-249a-452f-b5d4-8f613781ea74 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:15.382859 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d468ab-249a-452f-b5d4-8f613781ea74 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:15.411529 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d468ab-249a-452f-b5d4-8f613781ea74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:30:15.423334 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 176/340] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:30:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 17:30:15.424221 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8d539d-5e97-4bce-a50a-25b4a9eae789 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:15.427684 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-36d468ab-249a-452f-b5d4-8f613781ea74 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 17:30:15.427684 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:30:15.427684 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:30:15.427684 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:30:15.427684 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:30:15.427684 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:30:15.427684 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:30:15.427684 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:30:15.427684 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:30:15.427684 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:30:15.427684 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:30:15.427684 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:30:15.427684 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:30:15.427684 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:30:15.427684 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:30:15.427684 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:30:15.428574 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:30:15.428574 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:30:15.428574 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:30:15.428574 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:30:15.428574 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:30:15.428574 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:30:15.428574 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:30:15.428574 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:30:15.428574 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:30:15.428574 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:30:15.428574 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:30:15.428574 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:30:15.428574 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:30:15.428574 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:30:15.428574 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:30:15.428574 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:30:15.429352 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:30:15.429352 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:30:15.429352 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:30:15.429352 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:30:15.429352 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:30:15.429352 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:30:15.429352 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:30:15.429352 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:30:15.429352 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:30:15.429352 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:30:15.429352 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 17:30:15.429352 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:30:15.432262 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 165/341] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jun 16 17:30:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 458 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:30:15.441852 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d468ab-249a-452f-b5d4-8f613781ea74 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:15.544954 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:15.588551 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d468ab-249a-452f-b5d4-8f613781ea74 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:15.598855 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:15.630534 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14622c44-792f-49ce-84eb-22294e5ceea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:15.631396 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14622c44-792f-49ce-84eb-22294e5ceea4 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:15.631780 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14622c44-792f-49ce-84eb-22294e5ceea4 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:15.632151 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14622c44-792f-49ce-84eb-22294e5ceea4 None admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:15.633046 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14622c44-792f-49ce-84eb-22294e5ceea4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:15.633498 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14622c44-792f-49ce-84eb-22294e5ceea4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:15.635285 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14622c44-792f-49ce-84eb-22294e5ceea4 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:15.650195 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 166/342] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Jun 16 17:30:15 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:15.651314 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14622c44-792f-49ce-84eb-22294e5ceea4 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:15.658545 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8d539d-5e97-4bce-a50a-25b4a9eae789 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:15.698415 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8505bac-e4e6-4a9d-b81c-c3cde7884472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:15.699498 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8505bac-e4e6-4a9d-b81c-c3cde7884472 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:15.699951 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8505bac-e4e6-4a9d-b81c-c3cde7884472 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:15.700483 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8505bac-e4e6-4a9d-b81c-c3cde7884472 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:15.715026 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8505bac-e4e6-4a9d-b81c-c3cde7884472 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:30:15.726238 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-e8505bac-e4e6-4a9d-b81c-c3cde7884472 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:30:15.726238 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:30:15.726238 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:30:15.726238 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:30:15.726238 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:30:15.726238 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:30:15.726238 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:30:15.726238 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:30:15.726238 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:30:15.726238 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:30:15.726238 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:30:15.726238 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:30:15.726238 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:30:15.726238 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:30:15.726238 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:30:15.726238 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:30:15.727161 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:30:15.727161 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:30:15.727161 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:30:15.727161 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:30:15.727161 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:30:15.727161 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:30:15.727161 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:30:15.727161 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:30:15.727161 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:30:15.727161 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:30:15.727161 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:30:15.727161 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:30:15.727161 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:30:15.727161 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:30:15.727161 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:30:15.727161 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:30:15.727958 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:30:15.727958 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:30:15.727958 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:30:15.727958 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:30:15.727958 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:30:15.727958 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:30:15.727958 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:30:15.727958 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:30:15.727958 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:30:15.727958 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:30:15.727958 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:30:15.727958 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:30:15.730428 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 177/343] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jun 16 17:30:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:30:15.736402 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbbd9a5-fe1f-416f-b381-7dd58a564c03 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:15.737181 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbbd9a5-fe1f-416f-b381-7dd58a564c03 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:15.737676 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbbd9a5-fe1f-416f-b381-7dd58a564c03 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:15.739262 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 178/344] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:30:15 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:30:15.740248 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbbd9a5-fe1f-416f-b381-7dd58a564c03 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:15.775593 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14622c44-792f-49ce-84eb-22294e5ceea4 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:15.808824 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:15.824468 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dfe35b-79b4-42aa-9167-d9270dc894f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:15.825348 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dfe35b-79b4-42aa-9167-d9270dc894f4 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:15.825638 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dfe35b-79b4-42aa-9167-d9270dc894f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:15.825911 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dfe35b-79b4-42aa-9167-d9270dc894f4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:15.840628 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dfe35b-79b4-42aa-9167-d9270dc894f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:30:15.852201 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-c9dfe35b-79b4-42aa-9167-d9270dc894f4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 17:30:15.852201 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:30:15.852201 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:30:15.852201 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:30:15.852201 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:30:15.852201 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:30:15.852201 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:30:15.852201 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:30:15.852201 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:30:15.852201 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:30:15.852201 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:30:15.852201 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:30:15.852201 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:30:15.852201 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:30:15.852201 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:30:15.852201 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:30:15.853225 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:30:15.853225 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:30:15.853225 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:30:15.853225 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:30:15.853225 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:30:15.853225 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:30:15.853225 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:30:15.853225 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:30:15.853225 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:30:15.853225 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:30:15.853225 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:30:15.853225 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:30:15.853225 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:30:15.853225 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:30:15.853225 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:30:15.853225 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:30:15.854058 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:30:15.854058 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:30:15.854058 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:30:15.854058 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:30:15.854058 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:30:15.854058 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:30:15.854058 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:30:15.854058 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:30:15.854058 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:30:15.854058 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:30:15.854058 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 17:30:15.854058 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:30:15.858302 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 167/345] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Jun 16 17:30:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:30:15.859698 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dfe35b-79b4-42aa-9167-d9270dc894f4 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:15.873177 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbbd9a5-fe1f-416f-b381-7dd58a564c03 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:15.902327 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:15.916195 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59be49a5-378c-4771-bea3-89f6b74ff06f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:15.917505 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59be49a5-378c-4771-bea3-89f6b74ff06f None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:15.918022 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59be49a5-378c-4771-bea3-89f6b74ff06f None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:15.918502 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59be49a5-378c-4771-bea3-89f6b74ff06f None admin] PATH_INFO: `/v3/users/f1a92b3cef3d4ddfa56b6c7476124fb4` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:15.919945 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59be49a5-378c-4771-bea3-89f6b74ff06f None admin] RBAC: Authorizing `identity:get_user(user_id=f1a92b3cef3d4ddfa56b6c7476124fb4)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:15.921689 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59be49a5-378c-4771-bea3-89f6b74ff06f None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:15.924708 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 179/346] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jun 16 17:30:15 2022] GET /identity/v3/users/f1a92b3cef3d4ddfa56b6c7476124fb4 => generated 293 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:15.926226 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cf0ed3-cb0f-4985-a4ab-efb650a2b1f8 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:15.926730 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cf0ed3-cb0f-4985-a4ab-efb650a2b1f8 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:15.927215 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cf0ed3-cb0f-4985-a4ab-efb650a2b1f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:15.949370 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-e8cf0ed3-cb0f-4985-a4ab-efb650a2b1f8 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:30:15.950936 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-e8cf0ed3-cb0f-4985-a4ab-efb650a2b1f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:30:15.991737 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dfe35b-79b4-42aa-9167-d9270dc894f4 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:16.030517 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 180/347] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:30:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:30:16.030890 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7815baff-a9cf-4679-9a2b-23300f912f2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:16.031775 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7815baff-a9cf-4679-9a2b-23300f912f2b None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:16.032055 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7815baff-a9cf-4679-9a2b-23300f912f2b None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:16.032330 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7815baff-a9cf-4679-9a2b-23300f912f2b None admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:16.032899 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cf0ed3-cb0f-4985-a4ab-efb650a2b1f8 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:16.033160 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7815baff-a9cf-4679-9a2b-23300f912f2b None admin] RBAC: Adding query filter params (name=member) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:16.033463 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7815baff-a9cf-4679-9a2b-23300f912f2b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:16.034905 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7815baff-a9cf-4679-9a2b-23300f912f2b None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:16.049977 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 168/348] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Jun 16 17:30:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:16.049977 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7815baff-a9cf-4679-9a2b-23300f912f2b None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:16.122411 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:16.182990 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7815baff-a9cf-4679-9a2b-23300f912f2b None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:16.188869 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cf0ed3-cb0f-4985-a4ab-efb650a2b1f8 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:16.223866 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:16.237220 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8187bde7-b3da-43f3-a983-86d114e79592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:16.237830 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8187bde7-b3da-43f3-a983-86d114e79592 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:16.238118 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8187bde7-b3da-43f3-a983-86d114e79592 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:16.238401 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8187bde7-b3da-43f3-a983-86d114e79592 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:16.239359 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8187bde7-b3da-43f3-a983-86d114e79592 None admin] RBAC: Adding query filter params (role.id=9ed5382e55d649faa9195c4a9eb6fa57, user.id=f1a92b3cef3d4ddfa56b6c7476124fb4, scope.system=all) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:16.239680 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8187bde7-b3da-43f3-a983-86d114e79592 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:16.241406 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8187bde7-b3da-43f3-a983-86d114e79592 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:16.242782 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67a9035-522d-49b4-80e1-190f3299594e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:16.243972 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67a9035-522d-49b4-80e1-190f3299594e None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:16.244980 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67a9035-522d-49b4-80e1-190f3299594e None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:16.245785 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67a9035-522d-49b4-80e1-190f3299594e None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:16.246759 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67a9035-522d-49b4-80e1-190f3299594e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:16.247392 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67a9035-522d-49b4-80e1-190f3299594e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:16.249319 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67a9035-522d-49b4-80e1-190f3299594e None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:16.259939 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 181/350] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jun 16 17:30:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:16.260746 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 169/350] 10.222.0.42 () {56 vars in 1308 bytes} [Thu Jun 16 17:30:16 2022] GET /identity/v3/role_assignments?user.id=f1a92b3cef3d4ddfa56b6c7476124fb4&scope.system=all&role.id=9ed5382e55d649faa9195c4a9eb6fa57 => generated 510 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:16.261401 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67a9035-522d-49b4-80e1-190f3299594e None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:16.261695 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8187bde7-b3da-43f3-a983-86d114e79592 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:16.398302 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67a9035-522d-49b4-80e1-190f3299594e None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:16.443727 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c98aa6d-bbb8-4c39-8119-bf74d9d1a5ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:16.444890 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c98aa6d-bbb8-4c39-8119-bf74d9d1a5ae None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:16.445379 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c98aa6d-bbb8-4c39-8119-bf74d9d1a5ae None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:16.445873 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c98aa6d-bbb8-4c39-8119-bf74d9d1a5ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:16.513101 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:16.514573 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8187bde7-b3da-43f3-a983-86d114e79592 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:16.533875 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:16.534335 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c98aa6d-bbb8-4c39-8119-bf74d9d1a5ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:16.536411 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c98aa6d-bbb8-4c39-8119-bf74d9d1a5ae None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:16.583054 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef64c9f7-87dd-496e-bf45-7b81f612f457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:16.583602 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef64c9f7-87dd-496e-bf45-7b81f612f457 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:16.583893 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef64c9f7-87dd-496e-bf45-7b81f612f457 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:16.584175 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef64c9f7-87dd-496e-bf45-7b81f612f457 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:16.607296 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef64c9f7-87dd-496e-bf45-7b81f612f457 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:30:16.621514 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-ef64c9f7-87dd-496e-bf45-7b81f612f457 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:30:16.621514 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:30:16.621514 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:30:16.621514 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:30:16.621514 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:30:16.621514 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:30:16.621514 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:30:16.621514 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:30:16.621514 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:30:16.621514 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:30:16.621514 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:30:16.621514 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:30:16.621514 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:30:16.621514 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:30:16.621514 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:30:16.621514 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:30:16.622555 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:30:16.622555 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:30:16.622555 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:30:16.622555 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:30:16.622555 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:30:16.622555 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:30:16.622555 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:30:16.622555 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:30:16.622555 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:30:16.622555 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:30:16.622555 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:30:16.622555 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:30:16.622555 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:30:16.622555 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:30:16.622555 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:30:16.622555 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:30:16.623388 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:30:16.623388 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:30:16.623388 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:30:16.623388 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:30:16.623388 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:30:16.623388 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:30:16.623388 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:30:16.623388 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:30:16.623388 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:30:16.623388 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:30:16.623388 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:30:16.623388 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:30:16.623388 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 170/351] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jun 16 17:30:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 362 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:30:16.624605 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef64c9f7-87dd-496e-bf45-7b81f612f457 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:16.684312 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 182/352] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jun 16 17:30:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:30:16.686173 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c98aa6d-bbb8-4c39-8119-bf74d9d1a5ae None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:16.782788 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef64c9f7-87dd-496e-bf45-7b81f612f457 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:16.820921 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:16.836312 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab7d50c-2923-47f6-b247-f8ceef77e769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:16.837083 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab7d50c-2923-47f6-b247-f8ceef77e769 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:16.837352 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab7d50c-2923-47f6-b247-f8ceef77e769 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:16.837627 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab7d50c-2923-47f6-b247-f8ceef77e769 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:16.848848 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c98aa6d-bbb8-4c39-8119-bf74d9d1a5ae None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:16.899674 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4d07a5-04a6-465d-a466-7dd9627d4e89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:16.900462 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4d07a5-04a6-465d-a466-7dd9627d4e89 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:16.900684 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4d07a5-04a6-465d-a466-7dd9627d4e89 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:16.900972 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4d07a5-04a6-465d-a466-7dd9627d4e89 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:16.901923 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4d07a5-04a6-465d-a466-7dd9627d4e89 None admin] RBAC: Adding query filter params (name=member) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:16.902234 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4d07a5-04a6-465d-a466-7dd9627d4e89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:16.905306 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4d07a5-04a6-465d-a466-7dd9627d4e89 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:16.908431 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:16.915224 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 183/353] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Jun 16 17:30:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:16.916573 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b33140a-8c2f-43f9-8a9d-8d1da5c2e355 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:16.917027 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b33140a-8c2f-43f9-8a9d-8d1da5c2e355 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:16.917314 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b33140a-8c2f-43f9-8a9d-8d1da5c2e355 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:16.918562 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 184/354] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:30:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:30:16.919515 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b33140a-8c2f-43f9-8a9d-8d1da5c2e355 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:16.960895 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab7d50c-2923-47f6-b247-f8ceef77e769 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:16.962692 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab7d50c-2923-47f6-b247-f8ceef77e769 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:17.073029 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b33140a-8c2f-43f9-8a9d-8d1da5c2e355 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:17.118225 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 171/355] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jun 16 17:30:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 494 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:30:17.119420 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab7d50c-2923-47f6-b247-f8ceef77e769 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:17.123090 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d345c2b-f6a7-473c-98fc-6f5d558b01ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:17.125895 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d345c2b-f6a7-473c-98fc-6f5d558b01ae None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:17.125895 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d345c2b-f6a7-473c-98fc-6f5d558b01ae None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:17.125895 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d345c2b-f6a7-473c-98fc-6f5d558b01ae None admin] PATH_INFO: `/v3/projects/3be66087f3954ee4bac1dc53ea4d59ed` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:17.128267 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d345c2b-f6a7-473c-98fc-6f5d558b01ae None admin] RBAC: Authorizing `identity:get_project(project_id=3be66087f3954ee4bac1dc53ea4d59ed)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:17.130018 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d345c2b-f6a7-473c-98fc-6f5d558b01ae None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:17.132116 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 185/356] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jun 16 17:30:16 2022] GET /identity/v3/projects/3be66087f3954ee4bac1dc53ea4d59ed => generated 298 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:17.133766 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd634c93-de74-4e20-9943-9eb4807c8f6d None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:17.134204 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd634c93-de74-4e20-9943-9eb4807c8f6d None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:17.134624 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd634c93-de74-4e20-9943-9eb4807c8f6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:17.157746 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-bd634c93-de74-4e20-9943-9eb4807c8f6d None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:30:17.159200 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-bd634c93-de74-4e20-9943-9eb4807c8f6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:30:17.174940 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:17.225712 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:17.259752 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 186/357] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:30:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:30:17.262896 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd634c93-de74-4e20-9943-9eb4807c8f6d None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:17.272182 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab7d50c-2923-47f6-b247-f8ceef77e769 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:17.322922 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8624951e-5d74-4b96-977c-b053297426b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:17.323932 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8624951e-5d74-4b96-977c-b053297426b0 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:17.324845 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8624951e-5d74-4b96-977c-b053297426b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:17.325324 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8624951e-5d74-4b96-977c-b053297426b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:17.410793 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd634c93-de74-4e20-9943-9eb4807c8f6d None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:17.421469 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8624951e-5d74-4b96-977c-b053297426b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:17.423775 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8624951e-5d74-4b96-977c-b053297426b0 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:17.463015 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23df84c2-7873-48c1-a4c9-df47142423fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:17.463880 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23df84c2-7873-48c1-a4c9-df47142423fe None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:17.464178 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23df84c2-7873-48c1-a4c9-df47142423fe None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:17.464506 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23df84c2-7873-48c1-a4c9-df47142423fe None admin] PATH_INFO: `/v3/projects/3be66087f3954ee4bac1dc53ea4d59ed` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:17.465696 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23df84c2-7873-48c1-a4c9-df47142423fe None admin] RBAC: Authorizing `identity:get_project(project_id=3be66087f3954ee4bac1dc53ea4d59ed)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:17.467136 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23df84c2-7873-48c1-a4c9-df47142423fe None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:17.469586 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 187/358] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jun 16 17:30:17 2022] GET /identity/v3/projects/3be66087f3954ee4bac1dc53ea4d59ed => generated 298 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:17.470640 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23df84c2-7873-48c1-a4c9-df47142423fe None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:17.484470 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:17.514639 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:17.555597 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 172/359] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jun 16 17:30:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:30:17.557012 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55877aee-ae75-4f6a-b028-ca3e8b34f015 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:17.557365 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55877aee-ae75-4f6a-b028-ca3e8b34f015 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:17.557642 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55877aee-ae75-4f6a-b028-ca3e8b34f015 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:17.558937 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 173/360] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:30:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:30:17.559642 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55877aee-ae75-4f6a-b028-ca3e8b34f015 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:17.606629 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23df84c2-7873-48c1-a4c9-df47142423fe None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:17.652029 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520e7f01-8d3d-4ac6-80fd-9fd9bc6be46d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:17.652993 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520e7f01-8d3d-4ac6-80fd-9fd9bc6be46d None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:17.655093 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520e7f01-8d3d-4ac6-80fd-9fd9bc6be46d None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:17.655093 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520e7f01-8d3d-4ac6-80fd-9fd9bc6be46d None admin] PATH_INFO: `/v3/groups/27d023f6aba14910ae7171cb846daf74` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:17.660866 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520e7f01-8d3d-4ac6-80fd-9fd9bc6be46d None admin] RBAC: Authorizing `identity:get_group(group_id=27d023f6aba14910ae7171cb846daf74)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:17.662380 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520e7f01-8d3d-4ac6-80fd-9fd9bc6be46d None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:17.669306 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 188/361] 10.222.0.42 () {56 vars in 1180 bytes} [Thu Jun 16 17:30:17 2022] GET /identity/v3/groups/27d023f6aba14910ae7171cb846daf74 => generated 226 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:17.670043 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520e7f01-8d3d-4ac6-80fd-9fd9bc6be46d None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:17.706688 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55877aee-ae75-4f6a-b028-ca3e8b34f015 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:17.749859 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaa52e8-dda2-491f-a447-5de040e60716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:17.751499 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaa52e8-dda2-491f-a447-5de040e60716 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:17.751779 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaa52e8-dda2-491f-a447-5de040e60716 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:17.752051 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaa52e8-dda2-491f-a447-5de040e60716 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:17.772916 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aaa52e8-dda2-491f-a447-5de040e60716 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:30:17.778088 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:17.783144 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-2aaa52e8-dda2-491f-a447-5de040e60716 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 17:30:17.783144 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:30:17.783144 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:30:17.783144 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:30:17.783144 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:30:17.783144 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:30:17.783144 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:30:17.783144 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:30:17.783144 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:30:17.783144 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:30:17.783144 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:30:17.783144 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:30:17.783144 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:30:17.783144 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:30:17.783144 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:30:17.783144 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:30:17.784093 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:30:17.784093 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:30:17.784093 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:30:17.784093 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:30:17.784093 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:30:17.784093 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:30:17.784093 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:30:17.784093 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:30:17.784093 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:30:17.784093 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:30:17.784093 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:30:17.784093 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:30:17.784093 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:30:17.784093 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:30:17.784093 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:30:17.784093 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:30:17.785065 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:30:17.785065 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:30:17.785065 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:30:17.785065 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:30:17.785065 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:30:17.785065 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:30:17.785065 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:30:17.785065 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:30:17.785065 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:30:17.785065 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:30:17.785065 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 17:30:17.785065 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:30:17.785679 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 174/362] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Jun 16 17:30:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:30:17.787179 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521bb30d-80f1-4893-9a98-31232c5d28ed None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:17.787475 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521bb30d-80f1-4893-9a98-31232c5d28ed None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:17.787755 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521bb30d-80f1-4893-9a98-31232c5d28ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:17.808515 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-521bb30d-80f1-4893-9a98-31232c5d28ed None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:30:17.810038 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-521bb30d-80f1-4893-9a98-31232c5d28ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:30:17.816686 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520e7f01-8d3d-4ac6-80fd-9fd9bc6be46d None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:17.844141 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:17.861050 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8e28f9-bdd8-499e-b5a9-2eacb23967d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:17.861551 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8e28f9-bdd8-499e-b5a9-2eacb23967d2 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:17.861832 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8e28f9-bdd8-499e-b5a9-2eacb23967d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:17.862106 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8e28f9-bdd8-499e-b5a9-2eacb23967d2 None admin] PATH_INFO: `/v3/groups/27d023f6aba14910ae7171cb846daf74` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:17.863372 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8e28f9-bdd8-499e-b5a9-2eacb23967d2 None admin] RBAC: Authorizing `identity:get_group(group_id=27d023f6aba14910ae7171cb846daf74)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:17.864899 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8e28f9-bdd8-499e-b5a9-2eacb23967d2 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:17.866568 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 189/363] 10.222.0.42 () {56 vars in 1180 bytes} [Thu Jun 16 17:30:17 2022] GET /identity/v3/groups/27d023f6aba14910ae7171cb846daf74 => generated 226 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:17.867345 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8e28f9-bdd8-499e-b5a9-2eacb23967d2 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:17.912825 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 175/364] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:30:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:30:17.913528 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521bb30d-80f1-4893-9a98-31232c5d28ed None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:17.996060 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8e28f9-bdd8-499e-b5a9-2eacb23967d2 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:18.038456 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63daa48e-0480-4e11-9c45-6443228db334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:18.039260 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63daa48e-0480-4e11-9c45-6443228db334 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:18.039545 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63daa48e-0480-4e11-9c45-6443228db334 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:18.039819 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63daa48e-0480-4e11-9c45-6443228db334 None admin] PATH_INFO: `/v3/projects/d820cdcb6c0741e380190e6fe127e992` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:18.041993 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63daa48e-0480-4e11-9c45-6443228db334 None admin] RBAC: Authorizing `identity:get_project(project_id=d820cdcb6c0741e380190e6fe127e992)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:18.043442 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63daa48e-0480-4e11-9c45-6443228db334 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:18.045314 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 190/365] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jun 16 17:30:17 2022] GET /identity/v3/projects/d820cdcb6c0741e380190e6fe127e992 => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:18.045713 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521bb30d-80f1-4893-9a98-31232c5d28ed None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:18.046064 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63daa48e-0480-4e11-9c45-6443228db334 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:18.064058 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:18.092080 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363cf160-4dab-4433-aaf5-4c567e6a6505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:18.092827 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363cf160-4dab-4433-aaf5-4c567e6a6505 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:18.093114 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363cf160-4dab-4433-aaf5-4c567e6a6505 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:18.093392 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363cf160-4dab-4433-aaf5-4c567e6a6505 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:18.094463 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363cf160-4dab-4433-aaf5-4c567e6a6505 None admin] RBAC: Adding query filter params (group.id=27d023f6aba14910ae7171cb846daf74, role.id=0be2075946f54c8485f1c275f746478a, scope.project.id=3be66087f3954ee4bac1dc53ea4d59ed) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:18.094868 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363cf160-4dab-4433-aaf5-4c567e6a6505 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:18.096350 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363cf160-4dab-4433-aaf5-4c567e6a6505 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:18.107103 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 176/366] 10.222.0.42 () {56 vars in 1376 bytes} [Thu Jun 16 17:30:17 2022] GET /identity/v3/role_assignments?group.id=27d023f6aba14910ae7171cb846daf74&scope.project.id=3be66087f3954ee4bac1dc53ea4d59ed&role.id=0be2075946f54c8485f1c275f746478a => generated 260 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:18.108649 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363cf160-4dab-4433-aaf5-4c567e6a6505 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:18.156141 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:18.215997 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63daa48e-0480-4e11-9c45-6443228db334 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:18.266238 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300b3a25-8f12-4b04-81d7-49082e8a26bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:18.267326 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300b3a25-8f12-4b04-81d7-49082e8a26bc None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:18.267326 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300b3a25-8f12-4b04-81d7-49082e8a26bc None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:18.267669 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300b3a25-8f12-4b04-81d7-49082e8a26bc None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:18.269680 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300b3a25-8f12-4b04-81d7-49082e8a26bc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:18.269953 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300b3a25-8f12-4b04-81d7-49082e8a26bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:18.271523 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300b3a25-8f12-4b04-81d7-49082e8a26bc None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:18.282224 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 191/367] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jun 16 17:30:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:18.283541 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300b3a25-8f12-4b04-81d7-49082e8a26bc None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:18.294804 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363cf160-4dab-4433-aaf5-4c567e6a6505 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:18.350819 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41337e06-1095-47fe-8fd2-dfed5edbfe7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:18.351793 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41337e06-1095-47fe-8fd2-dfed5edbfe7e None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:18.352075 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41337e06-1095-47fe-8fd2-dfed5edbfe7e None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:18.352861 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41337e06-1095-47fe-8fd2-dfed5edbfe7e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:18.353933 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41337e06-1095-47fe-8fd2-dfed5edbfe7e None admin] RBAC: Adding query filter params (group.id=27d023f6aba14910ae7171cb846daf74, role.id=243fd0a11d94477986deb1dcfb3f31dd, scope.project.id=3be66087f3954ee4bac1dc53ea4d59ed) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:18.354056 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41337e06-1095-47fe-8fd2-dfed5edbfe7e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:18.355611 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41337e06-1095-47fe-8fd2-dfed5edbfe7e None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:18.365694 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 177/368] 10.222.0.42 () {56 vars in 1376 bytes} [Thu Jun 16 17:30:18 2022] GET /identity/v3/role_assignments?group.id=27d023f6aba14910ae7171cb846daf74&scope.project.id=3be66087f3954ee4bac1dc53ea4d59ed&role.id=243fd0a11d94477986deb1dcfb3f31dd => generated 260 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:18.366502 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41337e06-1095-47fe-8fd2-dfed5edbfe7e None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:18.390768 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:18.439452 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300b3a25-8f12-4b04-81d7-49082e8a26bc None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:18.493181 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fb2938-db46-4dab-86fa-40828aa0308d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:18.494656 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fb2938-db46-4dab-86fa-40828aa0308d None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:18.495002 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fb2938-db46-4dab-86fa-40828aa0308d None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:18.495348 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fb2938-db46-4dab-86fa-40828aa0308d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:18.506323 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:18.510342 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fb2938-db46-4dab-86fa-40828aa0308d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:30:18.520530 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-41fb2938-db46-4dab-86fa-40828aa0308d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:30:18.520530 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:30:18.520530 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:30:18.520530 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:30:18.520530 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:30:18.520530 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:30:18.520530 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:30:18.520530 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:30:18.520530 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:30:18.520530 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:30:18.520530 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:30:18.520530 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:30:18.520530 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:30:18.520530 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:30:18.520530 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:30:18.520530 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:30:18.522045 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:30:18.522045 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:30:18.522045 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:30:18.522045 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:30:18.522045 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:30:18.522045 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:30:18.522045 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:30:18.522045 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:30:18.522045 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:30:18.522045 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:30:18.522045 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:30:18.522045 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:30:18.522045 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:30:18.522045 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:30:18.522045 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:30:18.522045 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:30:18.524317 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:30:18.524317 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:30:18.524317 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:30:18.524317 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:30:18.524317 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:30:18.524317 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:30:18.524317 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:30:18.524317 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:30:18.524317 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:30:18.524317 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:30:18.524317 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:30:18.524317 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:30:18.524317 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 192/369] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Jun 16 17:30:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:30:18.526067 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fb2938-db46-4dab-86fa-40828aa0308d None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:18.542750 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41337e06-1095-47fe-8fd2-dfed5edbfe7e None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:18.627187 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f3f7d0-3cf3-4bb2-b2af-e973ef4b7806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:18.627187 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f3f7d0-3cf3-4bb2-b2af-e973ef4b7806 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:18.627187 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f3f7d0-3cf3-4bb2-b2af-e973ef4b7806 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:18.627187 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f3f7d0-3cf3-4bb2-b2af-e973ef4b7806 None admin] PATH_INFO: `/v3/groups/27d023f6aba14910ae7171cb846daf74` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:18.628365 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f3f7d0-3cf3-4bb2-b2af-e973ef4b7806 None admin] RBAC: Authorizing `identity:get_group(group_id=27d023f6aba14910ae7171cb846daf74)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:18.630189 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f3f7d0-3cf3-4bb2-b2af-e973ef4b7806 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:18.635881 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 178/370] 10.222.0.42 () {56 vars in 1180 bytes} [Thu Jun 16 17:30:18 2022] GET /identity/v3/groups/27d023f6aba14910ae7171cb846daf74 => generated 226 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:18.635881 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f3f7d0-3cf3-4bb2-b2af-e973ef4b7806 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:18.679286 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fb2938-db46-4dab-86fa-40828aa0308d None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:18.715528 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:18.727122 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f064bd3-07bc-47a5-9422-d5ec084160d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:18.727790 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f064bd3-07bc-47a5-9422-d5ec084160d1 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:18.728075 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f064bd3-07bc-47a5-9422-d5ec084160d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:18.728354 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f064bd3-07bc-47a5-9422-d5ec084160d1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:18.775472 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f3f7d0-3cf3-4bb2-b2af-e973ef4b7806 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:18.819330 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f064bd3-07bc-47a5-9422-d5ec084160d1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:18.820863 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f064bd3-07bc-47a5-9422-d5ec084160d1 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:18.827026 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e078ea6-ced4-482d-b0cb-9833abde5099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:18.827999 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e078ea6-ced4-482d-b0cb-9833abde5099 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:18.828636 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e078ea6-ced4-482d-b0cb-9833abde5099 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:18.829128 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e078ea6-ced4-482d-b0cb-9833abde5099 None admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:18.830616 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e078ea6-ced4-482d-b0cb-9833abde5099 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:18.831133 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e078ea6-ced4-482d-b0cb-9833abde5099 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:18.833560 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e078ea6-ced4-482d-b0cb-9833abde5099 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:18.846378 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 179/371] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jun 16 17:30:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:18.848164 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e078ea6-ced4-482d-b0cb-9833abde5099 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:18.874279 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:18.966429 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 193/372] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jun 16 17:30:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:30:18.967324 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f064bd3-07bc-47a5-9422-d5ec084160d1 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:19.016409 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e078ea6-ced4-482d-b0cb-9833abde5099 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:19.016790 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:19.051608 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db1301a-4136-4d9a-9454-6e7e933ef3a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:19.052272 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db1301a-4136-4d9a-9454-6e7e933ef3a0 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:19.052626 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db1301a-4136-4d9a-9454-6e7e933ef3a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:19.052865 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db1301a-4136-4d9a-9454-6e7e933ef3a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:19.053854 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db1301a-4136-4d9a-9454-6e7e933ef3a0 None admin] RBAC: Adding query filter params (group.id=27d023f6aba14910ae7171cb846daf74, role.id=0be2075946f54c8485f1c275f746478a, scope.project.id=d820cdcb6c0741e380190e6fe127e992) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:19.054172 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db1301a-4136-4d9a-9454-6e7e933ef3a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:19.056192 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db1301a-4136-4d9a-9454-6e7e933ef3a0 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:19.065208 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 180/373] 10.222.0.42 () {56 vars in 1376 bytes} [Thu Jun 16 17:30:18 2022] GET /identity/v3/role_assignments?group.id=27d023f6aba14910ae7171cb846daf74&scope.project.id=d820cdcb6c0741e380190e6fe127e992&role.id=0be2075946f54c8485f1c275f746478a => generated 260 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:19.065572 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db1301a-4136-4d9a-9454-6e7e933ef3a0 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:19.100628 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f064bd3-07bc-47a5-9422-d5ec084160d1 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:19.147006 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236a3dcf-0430-4bf6-99d9-b583c8c5bb3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:19.148043 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236a3dcf-0430-4bf6-99d9-b583c8c5bb3c None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:19.148578 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236a3dcf-0430-4bf6-99d9-b583c8c5bb3c None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:19.149059 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236a3dcf-0430-4bf6-99d9-b583c8c5bb3c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:19.202197 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:19.238605 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236a3dcf-0430-4bf6-99d9-b583c8c5bb3c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:19.240576 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236a3dcf-0430-4bf6-99d9-b583c8c5bb3c None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:19.276692 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:19.283066 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db1301a-4136-4d9a-9454-6e7e933ef3a0 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:19.363198 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 194/374] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jun 16 17:30:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:30:19.367897 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad09e592-4594-4701-a735-f4be440aa213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:19.367897 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad09e592-4594-4701-a735-f4be440aa213 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:19.368342 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad09e592-4594-4701-a735-f4be440aa213 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:19.368657 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad09e592-4594-4701-a735-f4be440aa213 None admin] PATH_INFO: `/v3/projects/d820cdcb6c0741e380190e6fe127e992` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:19.369627 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236a3dcf-0430-4bf6-99d9-b583c8c5bb3c None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:19.369878 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad09e592-4594-4701-a735-f4be440aa213 None admin] RBAC: Authorizing `identity:get_project(project_id=d820cdcb6c0741e380190e6fe127e992)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:19.371336 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad09e592-4594-4701-a735-f4be440aa213 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:19.373221 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 181/375] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jun 16 17:30:19 2022] GET /identity/v3/projects/d820cdcb6c0741e380190e6fe127e992 => generated 302 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:19.381392 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad09e592-4594-4701-a735-f4be440aa213 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:19.485527 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236a3dcf-0430-4bf6-99d9-b583c8c5bb3c None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:19.514279 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad09e592-4594-4701-a735-f4be440aa213 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:19.520729 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:19.527974 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85599b49-e615-4063-91ce-56515603a2e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:19.529185 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85599b49-e615-4063-91ce-56515603a2e9 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:19.529668 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85599b49-e615-4063-91ce-56515603a2e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:19.530138 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85599b49-e615-4063-91ce-56515603a2e9 None admin] PATH_INFO: `/v3/projects/f64791acc5674278bd1baac8430d9719` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:19.532909 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85599b49-e615-4063-91ce-56515603a2e9 None admin] RBAC: Authorizing `identity:get_project(project_id=f64791acc5674278bd1baac8430d9719)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:19.534568 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85599b49-e615-4063-91ce-56515603a2e9 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:19.536867 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 195/376] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jun 16 17:30:19 2022] GET /identity/v3/projects/f64791acc5674278bd1baac8430d9719 => generated 344 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:19.542709 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85599b49-e615-4063-91ce-56515603a2e9 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:19.568272 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371e8acb-5625-4d8c-b30d-5114536300dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:19.568272 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371e8acb-5625-4d8c-b30d-5114536300dc None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:19.568272 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371e8acb-5625-4d8c-b30d-5114536300dc None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:19.568272 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371e8acb-5625-4d8c-b30d-5114536300dc None admin] PATH_INFO: `/v3/groups/27d023f6aba14910ae7171cb846daf74` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:19.568703 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371e8acb-5625-4d8c-b30d-5114536300dc None admin] RBAC: Authorizing `identity:get_group(group_id=27d023f6aba14910ae7171cb846daf74)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:19.568703 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371e8acb-5625-4d8c-b30d-5114536300dc None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:19.568703 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 182/377] 10.222.0.42 () {56 vars in 1180 bytes} [Thu Jun 16 17:30:19 2022] GET /identity/v3/groups/27d023f6aba14910ae7171cb846daf74 => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:19.577074 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371e8acb-5625-4d8c-b30d-5114536300dc None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:19.629795 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:19.757618 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85599b49-e615-4063-91ce-56515603a2e9 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:19.806468 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903f23e6-dbad-4e35-89a3-2b6b47d6413e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:19.807689 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903f23e6-dbad-4e35-89a3-2b6b47d6413e None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:19.809185 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903f23e6-dbad-4e35-89a3-2b6b47d6413e None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:19.811101 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903f23e6-dbad-4e35-89a3-2b6b47d6413e None admin] PATH_INFO: `/v3/groups/2fa79991bc7342d3bd95a309ee21fdb0` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:19.823033 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903f23e6-dbad-4e35-89a3-2b6b47d6413e None admin] RBAC: Authorizing `identity:get_group(group_id=2fa79991bc7342d3bd95a309ee21fdb0)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:19.824830 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903f23e6-dbad-4e35-89a3-2b6b47d6413e None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:19.830204 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 196/378] 10.222.0.42 () {56 vars in 1180 bytes} [Thu Jun 16 17:30:19 2022] GET /identity/v3/groups/2fa79991bc7342d3bd95a309ee21fdb0 => generated 229 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:19.837948 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903f23e6-dbad-4e35-89a3-2b6b47d6413e None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:19.857499 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371e8acb-5625-4d8c-b30d-5114536300dc None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:19.953759 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc5c2a8-d17e-4f05-9cde-c9d5787b05d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:19.953964 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc5c2a8-d17e-4f05-9cde-c9d5787b05d9 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:19.954200 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc5c2a8-d17e-4f05-9cde-c9d5787b05d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:19.954423 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc5c2a8-d17e-4f05-9cde-c9d5787b05d9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:19.955268 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc5c2a8-d17e-4f05-9cde-c9d5787b05d9 None admin] RBAC: Adding query filter params (group.id=27d023f6aba14910ae7171cb846daf74, role.id=243fd0a11d94477986deb1dcfb3f31dd, scope.project.id=d820cdcb6c0741e380190e6fe127e992) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:19.955579 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc5c2a8-d17e-4f05-9cde-c9d5787b05d9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:19.957075 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc5c2a8-d17e-4f05-9cde-c9d5787b05d9 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:19.976404 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 183/379] 10.222.0.42 () {56 vars in 1376 bytes} [Thu Jun 16 17:30:19 2022] GET /identity/v3/role_assignments?group.id=27d023f6aba14910ae7171cb846daf74&scope.project.id=d820cdcb6c0741e380190e6fe127e992&role.id=243fd0a11d94477986deb1dcfb3f31dd => generated 260 bytes in 395 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:19.984846 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:20.033728 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903f23e6-dbad-4e35-89a3-2b6b47d6413e None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:20.101718 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1e74a8-6c51-4a78-aaa9-db8d87e8eff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:20.102749 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1e74a8-6c51-4a78-aaa9-db8d87e8eff6 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:20.102749 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1e74a8-6c51-4a78-aaa9-db8d87e8eff6 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:20.102749 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1e74a8-6c51-4a78-aaa9-db8d87e8eff6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:20.104276 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1e74a8-6c51-4a78-aaa9-db8d87e8eff6 None admin] RBAC: Adding query filter params (group.id=2fa79991bc7342d3bd95a309ee21fdb0, role.id=3ca2dcbbd29d4a31a3ab78f2299bc239, scope.project.id=f64791acc5674278bd1baac8430d9719) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:20.104276 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1e74a8-6c51-4a78-aaa9-db8d87e8eff6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:20.105650 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1e74a8-6c51-4a78-aaa9-db8d87e8eff6 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:20.120678 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 197/380] 10.222.0.42 () {56 vars in 1376 bytes} [Thu Jun 16 17:30:19 2022] GET /identity/v3/role_assignments?group.id=2fa79991bc7342d3bd95a309ee21fdb0&scope.project.id=f64791acc5674278bd1baac8430d9719&role.id=3ca2dcbbd29d4a31a3ab78f2299bc239 => generated 260 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:20.617195 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0c397d-8362-47a5-84d3-bdf84e0f9730 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:20.617566 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0c397d-8362-47a5-84d3-bdf84e0f9730 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:20.617742 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0c397d-8362-47a5-84d3-bdf84e0f9730 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:20.619263 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 184/381] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:30:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:30:20.627258 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d4c01f-b753-4bca-a02b-470ba5326daa None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:20.627258 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d4c01f-b753-4bca-a02b-470ba5326daa None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:20.627258 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d4c01f-b753-4bca-a02b-470ba5326daa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:20.662342 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-c6d4c01f-b753-4bca-a02b-470ba5326daa None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:30:20.663888 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-c6d4c01f-b753-4bca-a02b-470ba5326daa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:30:20.952676 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 198/382] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:30:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:30:20.979989 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0c397d-8362-47a5-84d3-bdf84e0f9730 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:21.270351 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1909bfc-5582-44a0-ade8-04a30f301ea1 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:21.271689 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1909bfc-5582-44a0-ade8-04a30f301ea1 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:21.272205 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1909bfc-5582-44a0-ade8-04a30f301ea1 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:21.280700 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 199/383] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:30:21 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:30:21.289840 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:21.290023 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c99e70-a01d-4131-9dda-1dfa75a4f4ce None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:21.290741 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c99e70-a01d-4131-9dda-1dfa75a4f4ce None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:21.291468 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c99e70-a01d-4131-9dda-1dfa75a4f4ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:21.316774 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0c397d-8362-47a5-84d3-bdf84e0f9730 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:21.349559 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-31c99e70-a01d-4131-9dda-1dfa75a4f4ce None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:30:21.351134 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-31c99e70-a01d-4131-9dda-1dfa75a4f4ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:30:21.356826 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6ce6b3-0626-4931-bca7-5f02666a75fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:21.357867 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6ce6b3-0626-4931-bca7-5f02666a75fe None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:21.358169 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6ce6b3-0626-4931-bca7-5f02666a75fe None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:21.358584 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6ce6b3-0626-4931-bca7-5f02666a75fe None admin] PATH_INFO: `/v3/roles/member` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:21.376466 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6ce6b3-0626-4931-bca7-5f02666a75fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:30:21.385645 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-fb6ce6b3-0626-4931-bca7-5f02666a75fe None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:30:21.385645 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:30:21.385645 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:30:21.385645 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:30:21.385645 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:30:21.385645 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:30:21.385645 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:30:21.385645 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:30:21.385645 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:30:21.385645 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:30:21.385645 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:30:21.385645 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:30:21.385645 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:30:21.385645 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:30:21.385645 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:30:21.385645 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:30:21.386621 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:30:21.386621 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:30:21.386621 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:30:21.386621 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:30:21.386621 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:30:21.386621 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:30:21.386621 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:30:21.386621 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:30:21.386621 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:30:21.386621 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:30:21.386621 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:30:21.386621 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:30:21.386621 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:30:21.386621 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:30:21.386621 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:30:21.386621 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:30:21.387406 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:30:21.387406 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:30:21.387406 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:30:21.387406 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:30:21.387406 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:30:21.387406 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:30:21.387406 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:30:21.387406 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:30:21.387406 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:30:21.387406 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:30:21.387406 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:30:21.387406 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:30:21.388236 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 185/384] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jun 16 17:30:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 408 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:30:21.395802 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6ce6b3-0626-4931-bca7-5f02666a75fe None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:21.565110 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:21.632095 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 200/385] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:30:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:30:21.706875 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c99e70-a01d-4131-9dda-1dfa75a4f4ce None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:21.834070 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6ce6b3-0626-4931-bca7-5f02666a75fe None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:21.878783 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ed0a41-d5fa-4d3b-930a-bbed9e9f9c07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:21.879468 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ed0a41-d5fa-4d3b-930a-bbed9e9f9c07 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:21.879758 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ed0a41-d5fa-4d3b-930a-bbed9e9f9c07 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:21.880040 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ed0a41-d5fa-4d3b-930a-bbed9e9f9c07 None admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:21.883740 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ed0a41-d5fa-4d3b-930a-bbed9e9f9c07 None admin] RBAC: Adding query filter params (name=member) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:21.884096 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ed0a41-d5fa-4d3b-930a-bbed9e9f9c07 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:21.885576 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ed0a41-d5fa-4d3b-930a-bbed9e9f9c07 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:21.896267 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 186/386] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Jun 16 17:30:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 501 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:21.901318 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ed0a41-d5fa-4d3b-930a-bbed9e9f9c07 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:21.916765 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:22.028908 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ed0a41-d5fa-4d3b-930a-bbed9e9f9c07 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:22.076001 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bacd82-74b6-46a8-ac6d-d0d0790a00b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:22.077270 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bacd82-74b6-46a8-ac6d-d0d0790a00b9 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:22.077695 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bacd82-74b6-46a8-ac6d-d0d0790a00b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:22.078672 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bacd82-74b6-46a8-ac6d-d0d0790a00b9 None admin] PATH_INFO: `/v3/groups/27d023f6aba14910ae7171cb846daf74` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:22.080159 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bacd82-74b6-46a8-ac6d-d0d0790a00b9 None admin] RBAC: Authorizing `identity:get_group(group_id=27d023f6aba14910ae7171cb846daf74)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:22.081963 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bacd82-74b6-46a8-ac6d-d0d0790a00b9 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:22.084326 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 187/387] 10.222.0.42 () {56 vars in 1180 bytes} [Thu Jun 16 17:30:21 2022] GET /identity/v3/groups/27d023f6aba14910ae7171cb846daf74 => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:30:22.085712 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e56bed0-b83e-445e-bc86-22fe0d15cd69 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:22.086257 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e56bed0-b83e-445e-bc86-22fe0d15cd69 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:22.086675 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e56bed0-b83e-445e-bc86-22fe0d15cd69 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:22.091941 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c99e70-a01d-4131-9dda-1dfa75a4f4ce None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:22.092398 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 188/388] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:30:22 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:30:22.093458 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e56bed0-b83e-445e-bc86-22fe0d15cd69 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:22.134345 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f18144-37bd-4948-869c-71aa7e9dd42e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:22.135245 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f18144-37bd-4948-869c-71aa7e9dd42e None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:22.135820 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f18144-37bd-4948-869c-71aa7e9dd42e None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:22.136196 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f18144-37bd-4948-869c-71aa7e9dd42e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:22.157636 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f18144-37bd-4948-869c-71aa7e9dd42e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:30:22.175735 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-92f18144-37bd-4948-869c-71aa7e9dd42e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 17:30:22.175735 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:30:22.175735 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:30:22.175735 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:30:22.175735 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:30:22.175735 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:30:22.175735 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:30:22.175735 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:30:22.175735 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:30:22.175735 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:30:22.175735 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:30:22.175735 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:30:22.175735 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:30:22.175735 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:30:22.175735 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:30:22.175735 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:30:22.176751 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:30:22.176751 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:30:22.176751 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:30:22.176751 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:30:22.176751 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:30:22.176751 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:30:22.176751 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:30:22.176751 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:30:22.176751 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:30:22.176751 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:30:22.176751 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:30:22.176751 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:30:22.176751 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:30:22.176751 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:30:22.176751 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:30:22.176751 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:30:22.177679 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:30:22.177679 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:30:22.177679 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:30:22.177679 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:30:22.177679 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:30:22.177679 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:30:22.177679 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:30:22.177679 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:30:22.177679 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:30:22.177679 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:30:22.177679 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 17:30:22.177679 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:30:22.180091 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 201/389] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Jun 16 17:30:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 476 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:30:22.181528 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21f7807-11cf-4bbd-9f13-6789abf27745 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:22.182036 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21f7807-11cf-4bbd-9f13-6789abf27745 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:22.182441 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21f7807-11cf-4bbd-9f13-6789abf27745 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:22.208954 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:22.209591 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-b21f7807-11cf-4bbd-9f13-6789abf27745 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:30:22.211201 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-b21f7807-11cf-4bbd-9f13-6789abf27745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:30:22.230490 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e56bed0-b83e-445e-bc86-22fe0d15cd69 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:22.276399 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f94f7f-f88a-4774-9681-2f9a44a35fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:22.276399 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f94f7f-f88a-4774-9681-2f9a44a35fe0 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:22.276399 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f94f7f-f88a-4774-9681-2f9a44a35fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:22.277393 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f94f7f-f88a-4774-9681-2f9a44a35fe0 None admin] PATH_INFO: `/v3/projects/3be66087f3954ee4bac1dc53ea4d59ed` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:22.279371 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f94f7f-f88a-4774-9681-2f9a44a35fe0 None admin] RBAC: Authorizing `identity:get_project(project_id=3be66087f3954ee4bac1dc53ea4d59ed)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:22.280927 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:22.281949 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f94f7f-f88a-4774-9681-2f9a44a35fe0 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:22.288639 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 189/390] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jun 16 17:30:22 2022] GET /identity/v3/projects/3be66087f3954ee4bac1dc53ea4d59ed => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:22.290051 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f94f7f-f88a-4774-9681-2f9a44a35fe0 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:22.319963 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 202/391] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:30:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:30:22.321360 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21f7807-11cf-4bbd-9f13-6789abf27745 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:22.416066 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f94f7f-f88a-4774-9681-2f9a44a35fe0 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:22.452222 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21f7807-11cf-4bbd-9f13-6789abf27745 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:22.461893 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b4a1b9-a44f-42df-9183-11a478ffca5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:22.463240 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b4a1b9-a44f-42df-9183-11a478ffca5a None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:22.464091 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b4a1b9-a44f-42df-9183-11a478ffca5a None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:22.464587 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b4a1b9-a44f-42df-9183-11a478ffca5a None admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:22.465779 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b4a1b9-a44f-42df-9183-11a478ffca5a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:22.466238 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b4a1b9-a44f-42df-9183-11a478ffca5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:22.469231 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b4a1b9-a44f-42df-9183-11a478ffca5a None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:22.481491 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 190/392] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jun 16 17:30:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:22.482585 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b4a1b9-a44f-42df-9183-11a478ffca5a None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:22.485835 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:22.499765 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4064c8fa-5249-4df7-953d-957edd504aa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:22.502258 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4064c8fa-5249-4df7-953d-957edd504aa7 None admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:22.502258 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4064c8fa-5249-4df7-953d-957edd504aa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:22.502258 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4064c8fa-5249-4df7-953d-957edd504aa7 None admin] PATH_INFO: `/v3/projects/3be66087f3954ee4bac1dc53ea4d59ed/groups/27d023f6aba14910ae7171cb846daf74/roles/0be2075946f54c8485f1c275f746478a` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:22.504530 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4064c8fa-5249-4df7-953d-957edd504aa7 None admin] RBAC: Authorizing `identity:create_grant(project_id=3be66087f3954ee4bac1dc53ea4d59ed, group_id=27d023f6aba14910ae7171cb846daf74, role_id=0be2075946f54c8485f1c275f746478a)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:22.507775 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4064c8fa-5249-4df7-953d-957edd504aa7 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:22.523967 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 203/393] 10.222.0.42 () {58 vars in 1442 bytes} [Thu Jun 16 17:30:22 2022] PUT /identity/v3/projects/3be66087f3954ee4bac1dc53ea4d59ed/groups/27d023f6aba14910ae7171cb846daf74/roles/0be2075946f54c8485f1c275f746478a => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:30:22.533479 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4064c8fa-5249-4df7-953d-957edd504aa7 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:22.584933 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:22.663011 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4064c8fa-5249-4df7-953d-957edd504aa7 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:22.684421 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b4a1b9-a44f-42df-9183-11a478ffca5a None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:22.707731 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43b7aa4-d534-4c4d-be2b-b6c1d7a5b864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:22.710423 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43b7aa4-d534-4c4d-be2b-b6c1d7a5b864 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:22.710703 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43b7aa4-d534-4c4d-be2b-b6c1d7a5b864 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:22.710981 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43b7aa4-d534-4c4d-be2b-b6c1d7a5b864 None admin] PATH_INFO: `/v3/groups/27d023f6aba14910ae7171cb846daf74` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:22.713468 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43b7aa4-d534-4c4d-be2b-b6c1d7a5b864 None admin] RBAC: Authorizing `identity:get_group(group_id=27d023f6aba14910ae7171cb846daf74)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:22.714930 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43b7aa4-d534-4c4d-be2b-b6c1d7a5b864 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:22.719612 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 204/394] 10.222.0.42 () {56 vars in 1180 bytes} [Thu Jun 16 17:30:22 2022] GET /identity/v3/groups/27d023f6aba14910ae7171cb846daf74 => generated 226 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:22.728497 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43b7aa4-d534-4c4d-be2b-b6c1d7a5b864 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:22.771171 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fe924a-eeb2-4f93-bab4-db3c03cd79d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:22.771989 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fe924a-eeb2-4f93-bab4-db3c03cd79d0 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:22.772280 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fe924a-eeb2-4f93-bab4-db3c03cd79d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:22.772579 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fe924a-eeb2-4f93-bab4-db3c03cd79d0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:22.798397 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fe924a-eeb2-4f93-bab4-db3c03cd79d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:30:22.806545 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-90fe924a-eeb2-4f93-bab4-db3c03cd79d0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:30:22.806545 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:30:22.806545 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:30:22.806545 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:30:22.806545 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:30:22.806545 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:30:22.806545 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:30:22.806545 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:30:22.806545 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:30:22.806545 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:30:22.806545 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:30:22.806545 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:30:22.806545 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:30:22.806545 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:30:22.806545 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:30:22.806545 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:30:22.807501 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:30:22.807501 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:30:22.807501 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:30:22.807501 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:30:22.807501 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:30:22.807501 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:30:22.807501 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:30:22.807501 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:30:22.807501 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:30:22.807501 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:30:22.807501 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:30:22.807501 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:30:22.807501 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:30:22.807501 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:30:22.807501 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:30:22.807501 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:30:22.808332 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:30:22.808332 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:30:22.808332 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:30:22.808332 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:30:22.808332 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:30:22.808332 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:30:22.808332 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:30:22.808332 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:30:22.808332 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:30:22.808332 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:30:22.808332 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:30:22.808332 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:30:22.808332 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 191/395] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jun 16 17:30:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:30:22.814732 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fe924a-eeb2-4f93-bab4-db3c03cd79d0 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:22.895014 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43b7aa4-d534-4c4d-be2b-b6c1d7a5b864 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:22.904107 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:22.924103 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:22.945248 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a125ecf7-ee36-43d1-a019-392becf8a143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:22.946148 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a125ecf7-ee36-43d1-a019-392becf8a143 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:22.946432 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a125ecf7-ee36-43d1-a019-392becf8a143 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:22.946718 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a125ecf7-ee36-43d1-a019-392becf8a143 None admin] PATH_INFO: `/v3/projects/3be66087f3954ee4bac1dc53ea4d59ed` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:22.948202 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a125ecf7-ee36-43d1-a019-392becf8a143 None admin] RBAC: Authorizing `identity:get_project(project_id=3be66087f3954ee4bac1dc53ea4d59ed)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:22.950443 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a125ecf7-ee36-43d1-a019-392becf8a143 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:22.952468 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 205/396] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jun 16 17:30:22 2022] GET /identity/v3/projects/3be66087f3954ee4bac1dc53ea4d59ed => generated 298 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:22.957688 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9f1e67-a9d9-490b-acde-983e509357c2 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:22.961460 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9f1e67-a9d9-490b-acde-983e509357c2 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:22.962278 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9f1e67-a9d9-490b-acde-983e509357c2 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:22.968354 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 206/397] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:30:22 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:30:22.970391 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9f1e67-a9d9-490b-acde-983e509357c2 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:22.982623 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fe924a-eeb2-4f93-bab4-db3c03cd79d0 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:23.024911 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a42c55c-cb94-4aee-be80-dbad67bc559c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:23.026017 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a42c55c-cb94-4aee-be80-dbad67bc559c None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:23.026504 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a42c55c-cb94-4aee-be80-dbad67bc559c None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:23.026984 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a42c55c-cb94-4aee-be80-dbad67bc559c None admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:23.027990 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a42c55c-cb94-4aee-be80-dbad67bc559c None admin] RBAC: Adding query filter params (name=member) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:23.028558 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a42c55c-cb94-4aee-be80-dbad67bc559c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:23.030221 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a42c55c-cb94-4aee-be80-dbad67bc559c None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:23.041361 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 192/398] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Jun 16 17:30:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:30:23.047605 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08db683-efee-4a7d-8fca-6a8b6b86e163 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:23.048139 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08db683-efee-4a7d-8fca-6a8b6b86e163 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:23.048655 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08db683-efee-4a7d-8fca-6a8b6b86e163 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:23.067789 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-d08db683-efee-4a7d-8fca-6a8b6b86e163 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:30:23.069396 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-d08db683-efee-4a7d-8fca-6a8b6b86e163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:30:23.096567 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9f1e67-a9d9-490b-acde-983e509357c2 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:23.134816 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ead9a4-fdb5-486b-8284-08e4f03981b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:23.135748 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ead9a4-fdb5-486b-8284-08e4f03981b9 None admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:23.136037 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ead9a4-fdb5-486b-8284-08e4f03981b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:23.136527 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ead9a4-fdb5-486b-8284-08e4f03981b9 None admin] PATH_INFO: `/v3/projects/3be66087f3954ee4bac1dc53ea4d59ed/groups/27d023f6aba14910ae7171cb846daf74/roles/243fd0a11d94477986deb1dcfb3f31dd` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:23.138937 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ead9a4-fdb5-486b-8284-08e4f03981b9 None admin] RBAC: Authorizing `identity:create_grant(project_id=3be66087f3954ee4bac1dc53ea4d59ed, group_id=27d023f6aba14910ae7171cb846daf74, role_id=243fd0a11d94477986deb1dcfb3f31dd)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:23.140314 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ead9a4-fdb5-486b-8284-08e4f03981b9 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:23.145400 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 193/399] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:30:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:30:23.146119 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08db683-efee-4a7d-8fca-6a8b6b86e163 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:23.164177 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 207/400] 10.222.0.42 () {58 vars in 1442 bytes} [Thu Jun 16 17:30:22 2022] PUT /identity/v3/projects/3be66087f3954ee4bac1dc53ea4d59ed/groups/27d023f6aba14910ae7171cb846daf74/roles/243fd0a11d94477986deb1dcfb3f31dd => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:30:23.165954 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d89f388-7cc7-4f7e-a1ab-aa75e6d647d8 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:23.166322 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d89f388-7cc7-4f7e-a1ab-aa75e6d647d8 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:23.166673 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d89f388-7cc7-4f7e-a1ab-aa75e6d647d8 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:23.173353 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 208/401] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:30:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:30:23.176675 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ff8137-db40-462b-9243-39868793d4d3 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:23.176971 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ff8137-db40-462b-9243-39868793d4d3 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:23.177487 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ff8137-db40-462b-9243-39868793d4d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:23.213331 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-94ff8137-db40-462b-9243-39868793d4d3 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:30:23.215517 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-94ff8137-db40-462b-9243-39868793d4d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:30:23.231444 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:23.264784 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:23.303273 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08db683-efee-4a7d-8fca-6a8b6b86e163 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:23.319957 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 209/402] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:30:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:30:23.326945 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ff8137-db40-462b-9243-39868793d4d3 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:23.364984 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e141f7b-20a6-42fd-bd66-ab9ae7997a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:23.365860 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e141f7b-20a6-42fd-bd66-ab9ae7997a55 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:23.366143 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e141f7b-20a6-42fd-bd66-ab9ae7997a55 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:23.366417 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e141f7b-20a6-42fd-bd66-ab9ae7997a55 None admin] PATH_INFO: `/v3/groups/27d023f6aba14910ae7171cb846daf74` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:23.367615 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e141f7b-20a6-42fd-bd66-ab9ae7997a55 None admin] RBAC: Authorizing `identity:get_group(group_id=27d023f6aba14910ae7171cb846daf74)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:23.369962 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e141f7b-20a6-42fd-bd66-ab9ae7997a55 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:23.371927 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 194/403] 10.222.0.42 () {56 vars in 1180 bytes} [Thu Jun 16 17:30:23 2022] GET /identity/v3/groups/27d023f6aba14910ae7171cb846daf74 => generated 226 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:23.377029 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e141f7b-20a6-42fd-bd66-ab9ae7997a55 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:23.457445 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ff8137-db40-462b-9243-39868793d4d3 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:23.498314 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7866e87d-573d-49b3-97bc-cc42f1433397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:23.500215 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7866e87d-573d-49b3-97bc-cc42f1433397 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:23.501238 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7866e87d-573d-49b3-97bc-cc42f1433397 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:23.501589 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7866e87d-573d-49b3-97bc-cc42f1433397 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:23.517838 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7866e87d-573d-49b3-97bc-cc42f1433397 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:30:23.518756 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e141f7b-20a6-42fd-bd66-ab9ae7997a55 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:23.526934 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-7866e87d-573d-49b3-97bc-cc42f1433397 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 17:30:23.526934 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:30:23.526934 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:30:23.526934 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:30:23.526934 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:30:23.526934 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:30:23.526934 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:30:23.526934 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:30:23.526934 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:30:23.526934 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:30:23.526934 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:30:23.526934 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:30:23.526934 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:30:23.526934 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:30:23.526934 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:30:23.526934 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:30:23.527858 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:30:23.527858 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:30:23.527858 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:30:23.527858 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:30:23.527858 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:30:23.527858 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:30:23.527858 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:30:23.527858 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:30:23.527858 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:30:23.527858 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:30:23.527858 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:30:23.527858 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:30:23.527858 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:30:23.527858 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:30:23.527858 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:30:23.527858 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:30:23.533160 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:30:23.533160 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:30:23.533160 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:30:23.533160 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:30:23.533160 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:30:23.533160 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:30:23.533160 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:30:23.533160 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:30:23.533160 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:30:23.533160 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:30:23.533160 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 17:30:23.533160 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:30:23.533160 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 210/404] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Jun 16 17:30:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:30:23.533160 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7866e87d-573d-49b3-97bc-cc42f1433397 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:23.553554 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:23.564343 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:23.569612 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ded0f9-ad84-4460-89af-fba102cfee10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:23.570727 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ded0f9-ad84-4460-89af-fba102cfee10 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:23.571228 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ded0f9-ad84-4460-89af-fba102cfee10 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:23.571671 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ded0f9-ad84-4460-89af-fba102cfee10 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:23.584617 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ded0f9-ad84-4460-89af-fba102cfee10 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:30:23.595042 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-14ded0f9-ad84-4460-89af-fba102cfee10 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:30:23.595042 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:30:23.595042 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:30:23.595042 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:30:23.595042 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:30:23.595042 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:30:23.595042 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:30:23.595042 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:30:23.595042 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:30:23.595042 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:30:23.595042 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:30:23.595042 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:30:23.595042 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:30:23.595042 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:30:23.595042 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:30:23.595042 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:30:23.595928 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:30:23.595928 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:30:23.595928 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:30:23.595928 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:30:23.595928 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:30:23.595928 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:30:23.595928 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:30:23.595928 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:30:23.595928 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:30:23.595928 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:30:23.595928 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:30:23.595928 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:30:23.595928 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:30:23.595928 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:30:23.595928 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:30:23.595928 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:30:23.596842 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:30:23.596842 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:30:23.596842 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:30:23.596842 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:30:23.596842 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:30:23.596842 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:30:23.596842 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:30:23.596842 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:30:23.596842 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:30:23.596842 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:30:23.596842 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:30:23.596842 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:30:23.604400 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 195/405] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Jun 16 17:30:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:30:23.605640 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ded0f9-ad84-4460-89af-fba102cfee10 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:23.656977 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7866e87d-573d-49b3-97bc-cc42f1433397 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:23.694909 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b000ea-f2d3-42b1-8def-df6c8063bf4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:23.695878 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b000ea-f2d3-42b1-8def-df6c8063bf4f None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:23.696233 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b000ea-f2d3-42b1-8def-df6c8063bf4f None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:23.696914 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b000ea-f2d3-42b1-8def-df6c8063bf4f None admin] PATH_INFO: `/v3/projects/d820cdcb6c0741e380190e6fe127e992` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:23.697980 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b000ea-f2d3-42b1-8def-df6c8063bf4f None admin] RBAC: Authorizing `identity:get_project(project_id=d820cdcb6c0741e380190e6fe127e992)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:23.699534 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b000ea-f2d3-42b1-8def-df6c8063bf4f None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:23.701324 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 211/406] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jun 16 17:30:23 2022] GET /identity/v3/projects/d820cdcb6c0741e380190e6fe127e992 => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:23.702279 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b000ea-f2d3-42b1-8def-df6c8063bf4f None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:23.717434 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ded0f9-ad84-4460-89af-fba102cfee10 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:23.754580 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db10e6df-d4b3-4a15-9e67-d9ca4d771ed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:23.755528 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db10e6df-d4b3-4a15-9e67-d9ca4d771ed1 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:23.756045 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db10e6df-d4b3-4a15-9e67-d9ca4d771ed1 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:23.756524 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db10e6df-d4b3-4a15-9e67-d9ca4d771ed1 None admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:23.757553 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db10e6df-d4b3-4a15-9e67-d9ca4d771ed1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:23.758050 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db10e6df-d4b3-4a15-9e67-d9ca4d771ed1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:23.759644 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db10e6df-d4b3-4a15-9e67-d9ca4d771ed1 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:23.769806 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 196/407] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jun 16 17:30:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:23.775185 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db10e6df-d4b3-4a15-9e67-d9ca4d771ed1 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:23.819040 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:23.824711 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:23.834658 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b000ea-f2d3-42b1-8def-df6c8063bf4f None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:23.880147 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bd6a4a-7eda-413a-a03e-edfdc2961029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:23.880984 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bd6a4a-7eda-413a-a03e-edfdc2961029 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:23.881276 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bd6a4a-7eda-413a-a03e-edfdc2961029 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:23.881557 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bd6a4a-7eda-413a-a03e-edfdc2961029 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:23.882488 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bd6a4a-7eda-413a-a03e-edfdc2961029 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:23.882819 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bd6a4a-7eda-413a-a03e-edfdc2961029 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:23.884213 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bd6a4a-7eda-413a-a03e-edfdc2961029 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:23.899285 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 212/408] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jun 16 17:30:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:23.899285 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bd6a4a-7eda-413a-a03e-edfdc2961029 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:23.899523 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db10e6df-d4b3-4a15-9e67-d9ca4d771ed1 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:23.934821 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6d034a-2cdf-469a-8002-b9c2e77bda66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:23.936118 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6d034a-2cdf-469a-8002-b9c2e77bda66 None admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:23.936636 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6d034a-2cdf-469a-8002-b9c2e77bda66 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:23.937107 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6d034a-2cdf-469a-8002-b9c2e77bda66 None admin] PATH_INFO: `/v3/projects/d820cdcb6c0741e380190e6fe127e992/groups/27d023f6aba14910ae7171cb846daf74/roles/0be2075946f54c8485f1c275f746478a` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:23.940052 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6d034a-2cdf-469a-8002-b9c2e77bda66 None admin] RBAC: Authorizing `identity:create_grant(project_id=d820cdcb6c0741e380190e6fe127e992, group_id=27d023f6aba14910ae7171cb846daf74, role_id=0be2075946f54c8485f1c275f746478a)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:23.941820 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6d034a-2cdf-469a-8002-b9c2e77bda66 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:23.966479 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 197/409] 10.222.0.42 () {58 vars in 1442 bytes} [Thu Jun 16 17:30:23 2022] PUT /identity/v3/projects/d820cdcb6c0741e380190e6fe127e992/groups/27d023f6aba14910ae7171cb846daf74/roles/0be2075946f54c8485f1c275f746478a => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:30:23.967660 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6d034a-2cdf-469a-8002-b9c2e77bda66 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:24.093684 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bd6a4a-7eda-413a-a03e-edfdc2961029 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:24.167020 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdac3b7-d819-4a70-b17b-1aee6a7c7648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:24.168277 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdac3b7-d819-4a70-b17b-1aee6a7c7648 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:24.169201 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdac3b7-d819-4a70-b17b-1aee6a7c7648 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:24.169810 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdac3b7-d819-4a70-b17b-1aee6a7c7648 None admin] PATH_INFO: `/v3/groups/27d023f6aba14910ae7171cb846daf74` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:24.171322 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdac3b7-d819-4a70-b17b-1aee6a7c7648 None admin] RBAC: Authorizing `identity:get_group(group_id=27d023f6aba14910ae7171cb846daf74)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:24.174943 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdac3b7-d819-4a70-b17b-1aee6a7c7648 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:24.177062 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 213/410] 10.222.0.42 () {56 vars in 1180 bytes} [Thu Jun 16 17:30:23 2022] GET /identity/v3/groups/27d023f6aba14910ae7171cb846daf74 => generated 226 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:24.182263 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6d034a-2cdf-469a-8002-b9c2e77bda66 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:24.183427 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdac3b7-d819-4a70-b17b-1aee6a7c7648 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:24.185008 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:24.215031 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:24.240202 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b30b1ec-ba71-40ad-a36a-d3db32d95cbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:24.241632 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b30b1ec-ba71-40ad-a36a-d3db32d95cbd None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:24.241632 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b30b1ec-ba71-40ad-a36a-d3db32d95cbd None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:24.241995 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b30b1ec-ba71-40ad-a36a-d3db32d95cbd None admin] PATH_INFO: `/v3/groups/2fa79991bc7342d3bd95a309ee21fdb0` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:24.243503 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b30b1ec-ba71-40ad-a36a-d3db32d95cbd None admin] RBAC: Authorizing `identity:get_group(group_id=2fa79991bc7342d3bd95a309ee21fdb0)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:24.245226 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b30b1ec-ba71-40ad-a36a-d3db32d95cbd None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:24.247175 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 198/411] 10.222.0.42 () {56 vars in 1180 bytes} [Thu Jun 16 17:30:23 2022] GET /identity/v3/groups/2fa79991bc7342d3bd95a309ee21fdb0 => generated 229 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:24.253367 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b30b1ec-ba71-40ad-a36a-d3db32d95cbd None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:24.315917 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdac3b7-d819-4a70-b17b-1aee6a7c7648 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:24.364636 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c45a808-9647-469c-810e-235b5614d2b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:24.365425 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c45a808-9647-469c-810e-235b5614d2b1 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:24.365707 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c45a808-9647-469c-810e-235b5614d2b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:24.365987 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c45a808-9647-469c-810e-235b5614d2b1 None admin] PATH_INFO: `/v3/projects/d820cdcb6c0741e380190e6fe127e992` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:24.367378 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c45a808-9647-469c-810e-235b5614d2b1 None admin] RBAC: Authorizing `identity:get_project(project_id=d820cdcb6c0741e380190e6fe127e992)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:24.368849 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c45a808-9647-469c-810e-235b5614d2b1 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:24.373346 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 214/412] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jun 16 17:30:24 2022] GET /identity/v3/projects/d820cdcb6c0741e380190e6fe127e992 => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:24.379514 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c45a808-9647-469c-810e-235b5614d2b1 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:24.460861 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b30b1ec-ba71-40ad-a36a-d3db32d95cbd None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:24.529544 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8280dd5-4914-47d6-a713-f91e5d301abe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:24.530413 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8280dd5-4914-47d6-a713-f91e5d301abe None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:24.530694 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8280dd5-4914-47d6-a713-f91e5d301abe None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:24.532059 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8280dd5-4914-47d6-a713-f91e5d301abe None admin] PATH_INFO: `/v3/projects/f64791acc5674278bd1baac8430d9719` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:24.532817 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8280dd5-4914-47d6-a713-f91e5d301abe None admin] RBAC: Authorizing `identity:get_project(project_id=f64791acc5674278bd1baac8430d9719)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:24.534388 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8280dd5-4914-47d6-a713-f91e5d301abe None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:24.541459 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 199/413] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jun 16 17:30:24 2022] GET /identity/v3/projects/f64791acc5674278bd1baac8430d9719 => generated 344 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:30:24.544178 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8280dd5-4914-47d6-a713-f91e5d301abe None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:24.562231 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:24.579099 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c45a808-9647-469c-810e-235b5614d2b1 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:24.641619 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:24.662033 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc5cf13-0528-4d7d-a340-d73474a90cbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:24.663161 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc5cf13-0528-4d7d-a340-d73474a90cbc None admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:24.666909 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc5cf13-0528-4d7d-a340-d73474a90cbc None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:24.667366 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc5cf13-0528-4d7d-a340-d73474a90cbc None admin] PATH_INFO: `/v3/projects/d820cdcb6c0741e380190e6fe127e992/groups/27d023f6aba14910ae7171cb846daf74/roles/243fd0a11d94477986deb1dcfb3f31dd` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:24.669896 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc5cf13-0528-4d7d-a340-d73474a90cbc None admin] RBAC: Authorizing `identity:create_grant(project_id=d820cdcb6c0741e380190e6fe127e992, group_id=27d023f6aba14910ae7171cb846daf74, role_id=243fd0a11d94477986deb1dcfb3f31dd)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:24.673380 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc5cf13-0528-4d7d-a340-d73474a90cbc None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:24.698965 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 215/414] 10.222.0.42 () {58 vars in 1442 bytes} [Thu Jun 16 17:30:24 2022] PUT /identity/v3/projects/d820cdcb6c0741e380190e6fe127e992/groups/27d023f6aba14910ae7171cb846daf74/roles/243fd0a11d94477986deb1dcfb3f31dd => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:30:24.768412 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8280dd5-4914-47d6-a713-f91e5d301abe None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:24.832421 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2e53f2-5218-4e38-b274-37b05079ed09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:24.833609 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2e53f2-5218-4e38-b274-37b05079ed09 None admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:24.834029 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2e53f2-5218-4e38-b274-37b05079ed09 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:24.834572 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2e53f2-5218-4e38-b274-37b05079ed09 None admin] PATH_INFO: `/v3/projects/f64791acc5674278bd1baac8430d9719/groups/2fa79991bc7342d3bd95a309ee21fdb0/roles/3ca2dcbbd29d4a31a3ab78f2299bc239` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:24.841879 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2e53f2-5218-4e38-b274-37b05079ed09 None admin] RBAC: Authorizing `identity:create_grant(project_id=f64791acc5674278bd1baac8430d9719, group_id=2fa79991bc7342d3bd95a309ee21fdb0, role_id=3ca2dcbbd29d4a31a3ab78f2299bc239)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:24.843552 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2e53f2-5218-4e38-b274-37b05079ed09 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:24.860996 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 200/415] 10.222.0.42 () {58 vars in 1442 bytes} [Thu Jun 16 17:30:24 2022] PUT /identity/v3/projects/f64791acc5674278bd1baac8430d9719/groups/2fa79991bc7342d3bd95a309ee21fdb0/roles/3ca2dcbbd29d4a31a3ab78f2299bc239 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:30:24.997922 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac61db3b-4565-4256-9b7d-2189ca6c5b20 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:24.998214 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac61db3b-4565-4256-9b7d-2189ca6c5b20 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:24.998501 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac61db3b-4565-4256-9b7d-2189ca6c5b20 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:25.000052 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 216/416] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:30:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:30:25.010498 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c508c7-9b3f-4f9a-9471-38a502466bc1 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:25.011076 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c508c7-9b3f-4f9a-9471-38a502466bc1 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:25.011557 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c508c7-9b3f-4f9a-9471-38a502466bc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:25.038263 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-04c508c7-9b3f-4f9a-9471-38a502466bc1 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:30:25.039824 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-04c508c7-9b3f-4f9a-9471-38a502466bc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:30:25.124309 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 201/417] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:30:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:30:25.161195 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac61db3b-4565-4256-9b7d-2189ca6c5b20 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:25.353829 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac61db3b-4565-4256-9b7d-2189ca6c5b20 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:25.398243 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8261d964-bb54-4ca3-93ba-ce4e667cf6bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:25.399112 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8261d964-bb54-4ca3-93ba-ce4e667cf6bf None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:25.399399 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8261d964-bb54-4ca3-93ba-ce4e667cf6bf None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:25.399674 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8261d964-bb54-4ca3-93ba-ce4e667cf6bf None admin] PATH_INFO: `/v3/roles/member` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:25.413489 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8261d964-bb54-4ca3-93ba-ce4e667cf6bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:30:25.422963 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-8261d964-bb54-4ca3-93ba-ce4e667cf6bf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:30:25.422963 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:30:25.422963 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:30:25.422963 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:30:25.422963 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:30:25.422963 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:30:25.422963 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:30:25.422963 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:30:25.422963 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:30:25.422963 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:30:25.422963 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:30:25.422963 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:30:25.422963 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:30:25.422963 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:30:25.422963 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:30:25.422963 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:30:25.424295 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:30:25.424295 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:30:25.424295 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:30:25.424295 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:30:25.424295 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:30:25.424295 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:30:25.424295 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:30:25.424295 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:30:25.424295 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:30:25.424295 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:30:25.424295 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:30:25.424295 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:30:25.424295 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:30:25.424295 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:30:25.424295 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:30:25.424295 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:30:25.425203 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:30:25.425203 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:30:25.425203 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:30:25.425203 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:30:25.425203 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:30:25.425203 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:30:25.425203 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:30:25.425203 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:30:25.425203 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:30:25.425203 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:30:25.425203 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:30:25.425203 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:30:25.425203 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 217/418] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jun 16 17:30:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:30:25.433816 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c508c7-9b3f-4f9a-9471-38a502466bc1 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:25.501225 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:25.619278 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42030a2d-17cb-4075-9cd1-1d9b995630ea None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:25.619658 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42030a2d-17cb-4075-9cd1-1d9b995630ea None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:25.619961 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42030a2d-17cb-4075-9cd1-1d9b995630ea None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:25.623648 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 218/419] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:30:25 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:30:25.633695 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bffda38-98bb-47de-bfbe-ea56237efabc None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:25.634312 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bffda38-98bb-47de-bfbe-ea56237efabc None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:25.634790 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bffda38-98bb-47de-bfbe-ea56237efabc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:25.672778 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c508c7-9b3f-4f9a-9471-38a502466bc1 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:25.674107 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-8bffda38-98bb-47de-bfbe-ea56237efabc None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:30:25.675738 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-8bffda38-98bb-47de-bfbe-ea56237efabc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:30:25.690719 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:25.719280 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d9101d-bff4-4aa4-9c62-32e257d89b93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:25.720261 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d9101d-bff4-4aa4-9c62-32e257d89b93 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:25.720744 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d9101d-bff4-4aa4-9c62-32e257d89b93 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:25.721609 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d9101d-bff4-4aa4-9c62-32e257d89b93 None admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:25.722718 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d9101d-bff4-4aa4-9c62-32e257d89b93 None admin] RBAC: Adding query filter params (name=member) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:25.723401 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d9101d-bff4-4aa4-9c62-32e257d89b93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:25.726159 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d9101d-bff4-4aa4-9c62-32e257d89b93 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:25.739560 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 202/420] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Jun 16 17:30:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:25.747659 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d9101d-bff4-4aa4-9c62-32e257d89b93 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:25.884727 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 219/421] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:30:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:30:25.907937 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bffda38-98bb-47de-bfbe-ea56237efabc None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:25.953783 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:25.987496 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d9101d-bff4-4aa4-9c62-32e257d89b93 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:26.045388 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bffda38-98bb-47de-bfbe-ea56237efabc None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:26.096401 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b507b1-4454-4957-83af-1a94a3297c2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:26.097430 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b507b1-4454-4957-83af-1a94a3297c2b None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:26.097917 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b507b1-4454-4957-83af-1a94a3297c2b None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:26.098520 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b507b1-4454-4957-83af-1a94a3297c2b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:26.114800 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf34f3b2-58c4-4aff-9ff1-a9b80cbb587d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:26.116498 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf34f3b2-58c4-4aff-9ff1-a9b80cbb587d None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:26.117620 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf34f3b2-58c4-4aff-9ff1-a9b80cbb587d None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:26.118107 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf34f3b2-58c4-4aff-9ff1-a9b80cbb587d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:26.144523 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:26.154927 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf34f3b2-58c4-4aff-9ff1-a9b80cbb587d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:30:26.169313 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-cf34f3b2-58c4-4aff-9ff1-a9b80cbb587d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 17:30:26.169313 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:30:26.169313 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:30:26.169313 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:30:26.169313 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:30:26.169313 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:30:26.169313 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:30:26.169313 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:30:26.169313 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:30:26.169313 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:30:26.169313 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:30:26.169313 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:30:26.169313 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:30:26.169313 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:30:26.169313 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:30:26.169313 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:30:26.170270 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:30:26.170270 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:30:26.170270 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:30:26.170270 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:30:26.170270 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:30:26.170270 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:30:26.170270 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:30:26.170270 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:30:26.170270 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:30:26.170270 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:30:26.170270 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:30:26.170270 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:30:26.170270 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:30:26.170270 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:30:26.170270 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:30:26.170270 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:30:26.171070 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:30:26.171070 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:30:26.171070 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:30:26.171070 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:30:26.171070 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:30:26.171070 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:30:26.171070 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:30:26.171070 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:30:26.171070 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:30:26.171070 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:30:26.171070 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 17:30:26.171070 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:30:26.174533 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 220/422] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Jun 16 17:30:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:30:26.187011 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf34f3b2-58c4-4aff-9ff1-a9b80cbb587d None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:26.293732 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b507b1-4454-4957-83af-1a94a3297c2b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:26.295660 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b507b1-4454-4957-83af-1a94a3297c2b None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:26.416277 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf34f3b2-58c4-4aff-9ff1-a9b80cbb587d None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:26.458234 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4a61a1-4a4b-4cee-ba9e-ed6ed79d4fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:26.459263 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4a61a1-4a4b-4cee-ba9e-ed6ed79d4fb4 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:26.459748 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4a61a1-4a4b-4cee-ba9e-ed6ed79d4fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:26.460274 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4a61a1-4a4b-4cee-ba9e-ed6ed79d4fb4 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:26.461383 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4a61a1-4a4b-4cee-ba9e-ed6ed79d4fb4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:26.462902 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4a61a1-4a4b-4cee-ba9e-ed6ed79d4fb4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:26.464852 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4a61a1-4a4b-4cee-ba9e-ed6ed79d4fb4 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:26.487984 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 221/423] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jun 16 17:30:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:30:26.490608 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:26.495136 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4a61a1-4a4b-4cee-ba9e-ed6ed79d4fb4 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:26.532001 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 203/424] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jun 16 17:30:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 784 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:30:26.537794 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b507b1-4454-4957-83af-1a94a3297c2b None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:26.588636 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:26.632742 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4a61a1-4a4b-4cee-ba9e-ed6ed79d4fb4 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:26.672742 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f56151-c4d9-4c35-a9d4-cd533348f55f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:26.673853 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f56151-c4d9-4c35-a9d4-cd533348f55f None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:26.674376 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f56151-c4d9-4c35-a9d4-cd533348f55f None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:26.674886 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f56151-c4d9-4c35-a9d4-cd533348f55f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:26.758668 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f56151-c4d9-4c35-a9d4-cd533348f55f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:26.760517 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f56151-c4d9-4c35-a9d4-cd533348f55f None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:26.836196 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:26.890918 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b507b1-4454-4957-83af-1a94a3297c2b None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:26.914163 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 222/425] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jun 16 17:30:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:30:26.918880 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee3647d-7e15-45d2-b84b-5670c8a521ab None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:26.918880 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee3647d-7e15-45d2-b84b-5670c8a521ab None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:26.918880 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee3647d-7e15-45d2-b84b-5670c8a521ab None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:26.918880 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 223/426] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:30:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:30:26.920667 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee3647d-7e15-45d2-b84b-5670c8a521ab None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:26.951818 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c439df-947d-4fc5-b804-dc2ed7a542c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:26.953036 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c439df-947d-4fc5-b804-dc2ed7a542c5 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:26.953311 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c439df-947d-4fc5-b804-dc2ed7a542c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:26.954069 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c439df-947d-4fc5-b804-dc2ed7a542c5 None admin] PATH_INFO: `/v3/projects/3be66087f3954ee4bac1dc53ea4d59ed` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:26.955538 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c439df-947d-4fc5-b804-dc2ed7a542c5 None admin] RBAC: Authorizing `identity:get_project(project_id=3be66087f3954ee4bac1dc53ea4d59ed)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:26.958000 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c439df-947d-4fc5-b804-dc2ed7a542c5 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:26.961082 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 204/427] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jun 16 17:30:26 2022] GET /identity/v3/projects/3be66087f3954ee4bac1dc53ea4d59ed => generated 298 bytes in 423 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:30:26.966762 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf4ea81-eea0-4824-b85b-78973e040e7d None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:26.967048 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf4ea81-eea0-4824-b85b-78973e040e7d None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:26.967326 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf4ea81-eea0-4824-b85b-78973e040e7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:26.990663 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-faf4ea81-eea0-4824-b85b-78973e040e7d None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:30:26.991857 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-faf4ea81-eea0-4824-b85b-78973e040e7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:30:27.062349 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee3647d-7e15-45d2-b84b-5670c8a521ab None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:27.069660 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:27.084662 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 205/428] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:30:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:30:27.085882 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf4ea81-eea0-4824-b85b-78973e040e7d None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:27.108213 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cb2cae-c93e-4628-a42e-c65eaf5d4454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:27.109132 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cb2cae-c93e-4628-a42e-c65eaf5d4454 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:27.109965 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cb2cae-c93e-4628-a42e-c65eaf5d4454 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:27.110249 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cb2cae-c93e-4628-a42e-c65eaf5d4454 None admin] PATH_INFO: `/v3/projects/3be66087f3954ee4bac1dc53ea4d59ed` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:27.111451 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cb2cae-c93e-4628-a42e-c65eaf5d4454 None admin] RBAC: Authorizing `identity:get_project(project_id=3be66087f3954ee4bac1dc53ea4d59ed)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:27.112948 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cb2cae-c93e-4628-a42e-c65eaf5d4454 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:27.115847 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 224/429] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jun 16 17:30:26 2022] GET /identity/v3/projects/3be66087f3954ee4bac1dc53ea4d59ed => generated 298 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:27.121223 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cb2cae-c93e-4628-a42e-c65eaf5d4454 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:27.163563 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:27.240764 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf4ea81-eea0-4824-b85b-78973e040e7d None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:27.274499 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cb2cae-c93e-4628-a42e-c65eaf5d4454 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:27.289909 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f99e7a-48f1-4589-8edc-ac08123f744e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:27.291404 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f99e7a-48f1-4589-8edc-ac08123f744e None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:27.292019 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f99e7a-48f1-4589-8edc-ac08123f744e None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:27.292663 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f99e7a-48f1-4589-8edc-ac08123f744e None admin] PATH_INFO: `/v3/groups/27d023f6aba14910ae7171cb846daf74` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:27.295021 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f99e7a-48f1-4589-8edc-ac08123f744e None admin] RBAC: Authorizing `identity:get_group(group_id=27d023f6aba14910ae7171cb846daf74)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:27.297899 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f99e7a-48f1-4589-8edc-ac08123f744e None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:27.306999 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 206/430] 10.222.0.42 () {56 vars in 1180 bytes} [Thu Jun 16 17:30:27 2022] GET /identity/v3/groups/27d023f6aba14910ae7171cb846daf74 => generated 226 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:27.306999 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f99e7a-48f1-4589-8edc-ac08123f744e None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:27.325662 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0211c080-7faf-4658-925a-9aa67258176b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:27.327170 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0211c080-7faf-4658-925a-9aa67258176b None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:27.327604 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0211c080-7faf-4658-925a-9aa67258176b None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:27.328023 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0211c080-7faf-4658-925a-9aa67258176b None admin] PATH_INFO: `/v3/groups/27d023f6aba14910ae7171cb846daf74` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:27.330909 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0211c080-7faf-4658-925a-9aa67258176b None admin] RBAC: Authorizing `identity:get_group(group_id=27d023f6aba14910ae7171cb846daf74)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:27.332778 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0211c080-7faf-4658-925a-9aa67258176b None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:27.335445 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 225/431] 10.222.0.42 () {56 vars in 1180 bytes} [Thu Jun 16 17:30:27 2022] GET /identity/v3/groups/27d023f6aba14910ae7171cb846daf74 => generated 226 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:27.336501 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0211c080-7faf-4658-925a-9aa67258176b None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:27.418826 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:27.496555 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f99e7a-48f1-4589-8edc-ac08123f744e None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:27.498862 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:27.505885 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0211c080-7faf-4658-925a-9aa67258176b None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:27.544900 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d88d7db-bfb8-40c3-af34-89aa3bbbf3f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:27.545122 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc89def-2b5c-43c3-b35c-c71747bb4073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:27.545560 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d88d7db-bfb8-40c3-af34-89aa3bbbf3f6 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:27.545824 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc89def-2b5c-43c3-b35c-c71747bb4073 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:27.545916 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d88d7db-bfb8-40c3-af34-89aa3bbbf3f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:27.546109 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc89def-2b5c-43c3-b35c-c71747bb4073 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:27.546201 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d88d7db-bfb8-40c3-af34-89aa3bbbf3f6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:27.546385 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc89def-2b5c-43c3-b35c-c71747bb4073 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:27.546994 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d88d7db-bfb8-40c3-af34-89aa3bbbf3f6 None admin] RBAC: Adding query filter params (group.id=27d023f6aba14910ae7171cb846daf74, role.id=0be2075946f54c8485f1c275f746478a, scope.project.id=3be66087f3954ee4bac1dc53ea4d59ed) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:27.547313 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d88d7db-bfb8-40c3-af34-89aa3bbbf3f6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:27.548780 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d88d7db-bfb8-40c3-af34-89aa3bbbf3f6 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:27.554718 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 226/432] 10.222.0.42 () {56 vars in 1376 bytes} [Thu Jun 16 17:30:27 2022] GET /identity/v3/role_assignments?group.id=27d023f6aba14910ae7171cb846daf74&scope.project.id=3be66087f3954ee4bac1dc53ea4d59ed&role.id=0be2075946f54c8485f1c275f746478a => generated 611 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:27.555507 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d88d7db-bfb8-40c3-af34-89aa3bbbf3f6 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:27.556901 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc89def-2b5c-43c3-b35c-c71747bb4073 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:30:27.565517 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-5bc89def-2b5c-43c3-b35c-c71747bb4073 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:30:27.565517 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:30:27.565517 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:30:27.565517 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:30:27.565517 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:30:27.565517 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:30:27.565517 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:30:27.565517 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:30:27.565517 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:30:27.565517 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:30:27.565517 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:30:27.565517 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:30:27.565517 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:30:27.565517 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:30:27.565517 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:30:27.565517 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:30:27.566639 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:30:27.566639 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:30:27.566639 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:30:27.566639 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:30:27.566639 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:30:27.566639 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:30:27.566639 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:30:27.566639 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:30:27.566639 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:30:27.566639 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:30:27.566639 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:30:27.566639 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:30:27.566639 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:30:27.566639 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:30:27.566639 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:30:27.566639 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:30:27.567438 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:30:27.567438 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:30:27.567438 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:30:27.567438 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:30:27.567438 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:30:27.567438 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:30:27.567438 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:30:27.567438 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:30:27.567438 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:30:27.567438 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:30:27.567438 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:30:27.567438 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:30:27.567438 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 207/433] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jun 16 17:30:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:30:27.575096 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ab115f-3bd6-42fe-bf82-45d9fb5e9569 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:27.575096 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ab115f-3bd6-42fe-bf82-45d9fb5e9569 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:27.575096 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ab115f-3bd6-42fe-bf82-45d9fb5e9569 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:27.575096 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 208/434] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:30:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:30:27.575096 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09079b8-7a3a-4825-b5b6-2b4d6e0b26bc None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:27.575096 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09079b8-7a3a-4825-b5b6-2b4d6e0b26bc None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:27.575587 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09079b8-7a3a-4825-b5b6-2b4d6e0b26bc None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:27.577340 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 209/435] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:30:27 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:30:27.579181 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09079b8-7a3a-4825-b5b6-2b4d6e0b26bc None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:27.695928 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09079b8-7a3a-4825-b5b6-2b4d6e0b26bc None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:27.700006 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d88d7db-bfb8-40c3-af34-89aa3bbbf3f6 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:27.714478 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:27.736333 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535bfba6-aa45-4aad-9e2e-fb4000b3d120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:27.737045 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535bfba6-aa45-4aad-9e2e-fb4000b3d120 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:27.737333 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535bfba6-aa45-4aad-9e2e-fb4000b3d120 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:27.737609 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535bfba6-aa45-4aad-9e2e-fb4000b3d120 None admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:27.738375 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535bfba6-aa45-4aad-9e2e-fb4000b3d120 None admin] RBAC: Adding query filter params (name=member) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:27.738685 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535bfba6-aa45-4aad-9e2e-fb4000b3d120 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:27.740077 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535bfba6-aa45-4aad-9e2e-fb4000b3d120 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:27.744416 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5cff84-317f-46df-9e56-7f40d04324af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:27.744874 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5cff84-317f-46df-9e56-7f40d04324af None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:27.745160 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5cff84-317f-46df-9e56-7f40d04324af None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:27.745437 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5cff84-317f-46df-9e56-7f40d04324af None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:27.746273 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5cff84-317f-46df-9e56-7f40d04324af None admin] RBAC: Adding query filter params (group.id=27d023f6aba14910ae7171cb846daf74, role.id=243fd0a11d94477986deb1dcfb3f31dd, scope.project.id=3be66087f3954ee4bac1dc53ea4d59ed) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:27.746641 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5cff84-317f-46df-9e56-7f40d04324af None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:27.748039 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5cff84-317f-46df-9e56-7f40d04324af None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:27.751783 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 210/436] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Jun 16 17:30:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:27.752984 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa0613d-e529-4a53-9b01-b49f16f9ffaf None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:27.754051 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa0613d-e529-4a53-9b01-b49f16f9ffaf None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:27.754051 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa0613d-e529-4a53-9b01-b49f16f9ffaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:27.758162 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 227/437] 10.222.0.42 () {56 vars in 1376 bytes} [Thu Jun 16 17:30:27 2022] GET /identity/v3/role_assignments?group.id=27d023f6aba14910ae7171cb846daf74&scope.project.id=3be66087f3954ee4bac1dc53ea4d59ed&role.id=243fd0a11d94477986deb1dcfb3f31dd => generated 611 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:27.759409 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b75b2d5-f4e6-4cb6-8c7a-7f2c01b2a8dd None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:27.759679 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b75b2d5-f4e6-4cb6-8c7a-7f2c01b2a8dd None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:27.759958 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b75b2d5-f4e6-4cb6-8c7a-7f2c01b2a8dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:27.778482 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-faa0613d-e529-4a53-9b01-b49f16f9ffaf None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:30:27.779710 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-faa0613d-e529-4a53-9b01-b49f16f9ffaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:30:27.782984 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-8b75b2d5-f4e6-4cb6-8c7a-7f2c01b2a8dd None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:30:27.784213 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-8b75b2d5-f4e6-4cb6-8c7a-7f2c01b2a8dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:30:27.811383 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:27.898166 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 228/438] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:30:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:30:27.902152 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b75b2d5-f4e6-4cb6-8c7a-7f2c01b2a8dd None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:27.907956 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 211/439] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:30:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:30:27.926924 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa0613d-e529-4a53-9b01-b49f16f9ffaf None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:28.046466 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b75b2d5-f4e6-4cb6-8c7a-7f2c01b2a8dd None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:28.057703 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa0613d-e529-4a53-9b01-b49f16f9ffaf None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:28.073648 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:28.086085 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0debf292-1e9a-4625-b03b-cb7483c26c2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:28.086759 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0debf292-1e9a-4625-b03b-cb7483c26c2f None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:28.087287 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0debf292-1e9a-4625-b03b-cb7483c26c2f None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:28.087486 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0debf292-1e9a-4625-b03b-cb7483c26c2f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:28.101513 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33da09d7-fe55-4f58-9c0c-af8ce912f19f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:28.102328 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33da09d7-fe55-4f58-9c0c-af8ce912f19f None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:28.102611 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33da09d7-fe55-4f58-9c0c-af8ce912f19f None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:28.102883 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33da09d7-fe55-4f58-9c0c-af8ce912f19f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:28.117699 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33da09d7-fe55-4f58-9c0c-af8ce912f19f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:30:28.121944 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:28.131978 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-33da09d7-fe55-4f58-9c0c-af8ce912f19f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 17:30:28.131978 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:30:28.131978 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:30:28.131978 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:30:28.131978 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:30:28.131978 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:30:28.131978 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:30:28.131978 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:30:28.131978 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:30:28.131978 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:30:28.131978 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:30:28.131978 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:30:28.131978 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:30:28.131978 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:30:28.131978 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:30:28.131978 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:30:28.133027 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:30:28.133027 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:30:28.133027 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:30:28.133027 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:30:28.133027 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:30:28.133027 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:30:28.133027 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:30:28.133027 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:30:28.133027 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:30:28.133027 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:30:28.133027 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:30:28.133027 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:30:28.133027 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:30:28.133027 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:30:28.133027 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:30:28.133027 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:30:28.133813 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:30:28.133813 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:30:28.133813 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:30:28.133813 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:30:28.133813 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:30:28.133813 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:30:28.133813 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:30:28.133813 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:30:28.133813 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:30:28.133813 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:30:28.133813 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 17:30:28.133813 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:30:28.133813 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 212/440] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Jun 16 17:30:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:30:28.134465 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33da09d7-fe55-4f58-9c0c-af8ce912f19f None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:28.181381 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0debf292-1e9a-4625-b03b-cb7483c26c2f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:28.183758 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0debf292-1e9a-4625-b03b-cb7483c26c2f None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:28.264611 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33da09d7-fe55-4f58-9c0c-af8ce912f19f None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:28.296669 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 229/441] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jun 16 17:30:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:30:28.298045 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0debf292-1e9a-4625-b03b-cb7483c26c2f None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:28.299808 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9344c0-6c3e-41a5-91c2-c61e2f683172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:28.300784 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9344c0-6c3e-41a5-91c2-c61e2f683172 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:28.301214 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9344c0-6c3e-41a5-91c2-c61e2f683172 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:28.301494 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9344c0-6c3e-41a5-91c2-c61e2f683172 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:28.312083 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9344c0-6c3e-41a5-91c2-c61e2f683172 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:30:28.319775 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-8d9344c0-6c3e-41a5-91c2-c61e2f683172 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:30:28.319775 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:30:28.319775 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:30:28.319775 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:30:28.319775 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:30:28.319775 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:30:28.319775 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:30:28.319775 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:30:28.319775 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:30:28.319775 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:30:28.319775 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:30:28.319775 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:30:28.319775 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:30:28.319775 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:30:28.319775 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:30:28.319775 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:30:28.320734 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:30:28.320734 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:30:28.320734 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:30:28.320734 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:30:28.320734 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:30:28.320734 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:30:28.320734 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:30:28.320734 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:30:28.320734 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:30:28.320734 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:30:28.320734 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:30:28.320734 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:30:28.320734 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:30:28.320734 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:30:28.320734 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:30:28.320734 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:30:28.321529 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:30:28.321529 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:30:28.321529 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:30:28.321529 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:30:28.321529 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:30:28.321529 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:30:28.321529 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:30:28.321529 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:30:28.321529 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:30:28.321529 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:30:28.321529 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:30:28.321529 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:30:28.321529 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 213/442] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Jun 16 17:30:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:30:28.322181 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9344c0-6c3e-41a5-91c2-c61e2f683172 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:28.355605 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:28.357780 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:28.401877 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0debf292-1e9a-4625-b03b-cb7483c26c2f None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:28.428381 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9344c0-6c3e-41a5-91c2-c61e2f683172 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:28.436314 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ca8b70-c6fc-4632-9c79-cf37a2cc7762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:28.438340 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ca8b70-c6fc-4632-9c79-cf37a2cc7762 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:28.438340 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ca8b70-c6fc-4632-9c79-cf37a2cc7762 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:28.438340 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ca8b70-c6fc-4632-9c79-cf37a2cc7762 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:28.438594 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ca8b70-c6fc-4632-9c79-cf37a2cc7762 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:28.438826 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ca8b70-c6fc-4632-9c79-cf37a2cc7762 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:28.440227 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ca8b70-c6fc-4632-9c79-cf37a2cc7762 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:28.448812 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 230/443] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jun 16 17:30:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:28.449651 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ca8b70-c6fc-4632-9c79-cf37a2cc7762 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:28.463365 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb7b72c-186c-441d-9a74-a100d8b2d7ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:28.464295 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb7b72c-186c-441d-9a74-a100d8b2d7ed None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:28.464609 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb7b72c-186c-441d-9a74-a100d8b2d7ed None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:28.464888 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb7b72c-186c-441d-9a74-a100d8b2d7ed None admin] PATH_INFO: `/v3/projects/d820cdcb6c0741e380190e6fe127e992` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:28.466073 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb7b72c-186c-441d-9a74-a100d8b2d7ed None admin] RBAC: Authorizing `identity:get_project(project_id=d820cdcb6c0741e380190e6fe127e992)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:28.467531 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb7b72c-186c-441d-9a74-a100d8b2d7ed None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:28.469924 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 214/444] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jun 16 17:30:28 2022] GET /identity/v3/projects/d820cdcb6c0741e380190e6fe127e992 => generated 302 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:28.470699 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb7b72c-186c-441d-9a74-a100d8b2d7ed None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:28.555180 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ca8b70-c6fc-4632-9c79-cf37a2cc7762 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:28.570046 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:28.572920 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:28.575223 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb7b72c-186c-441d-9a74-a100d8b2d7ed None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:28.590074 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26681456-cdce-4ce1-b9c3-55c8d4e12dac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:28.590741 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26681456-cdce-4ce1-b9c3-55c8d4e12dac None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:28.591016 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26681456-cdce-4ce1-b9c3-55c8d4e12dac None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:28.591292 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26681456-cdce-4ce1-b9c3-55c8d4e12dac None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:28.592022 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26681456-cdce-4ce1-b9c3-55c8d4e12dac None admin] RBAC: Adding query filter params (name=admin) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:28.592337 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26681456-cdce-4ce1-b9c3-55c8d4e12dac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:28.594253 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26681456-cdce-4ce1-b9c3-55c8d4e12dac None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:28.602342 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 231/445] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jun 16 17:30:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:28.603128 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26681456-cdce-4ce1-b9c3-55c8d4e12dac None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:28.610404 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7824ac19-f1f9-49f9-aa44-524c7984e456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:28.611060 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7824ac19-f1f9-49f9-aa44-524c7984e456 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:28.611349 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7824ac19-f1f9-49f9-aa44-524c7984e456 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:28.611629 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7824ac19-f1f9-49f9-aa44-524c7984e456 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:28.680880 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7824ac19-f1f9-49f9-aa44-524c7984e456 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:28.682355 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7824ac19-f1f9-49f9-aa44-524c7984e456 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:28.705371 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26681456-cdce-4ce1-b9c3-55c8d4e12dac None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:28.739506 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5288c5c-1b56-4c4e-b647-f06b125bad0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:28.740355 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5288c5c-1b56-4c4e-b647-f06b125bad0b None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:28.740661 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5288c5c-1b56-4c4e-b647-f06b125bad0b None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:28.740933 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5288c5c-1b56-4c4e-b647-f06b125bad0b None admin] PATH_INFO: `/v3/groups/27d023f6aba14910ae7171cb846daf74` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:28.742108 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5288c5c-1b56-4c4e-b647-f06b125bad0b None admin] RBAC: Authorizing `identity:get_group(group_id=27d023f6aba14910ae7171cb846daf74)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:28.743531 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5288c5c-1b56-4c4e-b647-f06b125bad0b None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:28.745170 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 232/446] 10.222.0.42 () {56 vars in 1180 bytes} [Thu Jun 16 17:30:28 2022] GET /identity/v3/groups/27d023f6aba14910ae7171cb846daf74 => generated 226 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:28.745938 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5288c5c-1b56-4c4e-b647-f06b125bad0b None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:28.786356 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:28.792162 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 215/447] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jun 16 17:30:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:30:28.793547 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7824ac19-f1f9-49f9-aa44-524c7984e456 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:28.802474 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:28.855058 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5288c5c-1b56-4c4e-b647-f06b125bad0b None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:28.894729 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39274bf8-0b6d-42c5-8057-f37120ee4b94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:28.895646 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39274bf8-0b6d-42c5-8057-f37120ee4b94 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:28.895646 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39274bf8-0b6d-42c5-8057-f37120ee4b94 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:28.895950 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39274bf8-0b6d-42c5-8057-f37120ee4b94 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:28.896032 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7824ac19-f1f9-49f9-aa44-524c7984e456 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:28.932416 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fead2a-5f5b-42af-918d-861469ea9609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:28.933201 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fead2a-5f5b-42af-918d-861469ea9609 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:28.933574 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fead2a-5f5b-42af-918d-861469ea9609 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:28.934066 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fead2a-5f5b-42af-918d-861469ea9609 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:28.934972 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fead2a-5f5b-42af-918d-861469ea9609 None admin] RBAC: Adding query filter params (group.id=27d023f6aba14910ae7171cb846daf74, role.id=0be2075946f54c8485f1c275f746478a, scope.project.id=d820cdcb6c0741e380190e6fe127e992) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:28.935395 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fead2a-5f5b-42af-918d-861469ea9609 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:28.937118 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fead2a-5f5b-42af-918d-861469ea9609 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:28.944144 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 216/448] 10.222.0.42 () {56 vars in 1376 bytes} [Thu Jun 16 17:30:28 2022] GET /identity/v3/role_assignments?group.id=27d023f6aba14910ae7171cb846daf74&scope.project.id=d820cdcb6c0741e380190e6fe127e992&role.id=0be2075946f54c8485f1c275f746478a => generated 611 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:28.944979 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fead2a-5f5b-42af-918d-861469ea9609 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:28.977071 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39274bf8-0b6d-42c5-8057-f37120ee4b94 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:28.978574 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39274bf8-0b6d-42c5-8057-f37120ee4b94 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:29.027602 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:29.040673 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:29.076359 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fead2a-5f5b-42af-918d-861469ea9609 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:29.112192 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 233/449] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jun 16 17:30:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:30:29.117601 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39274bf8-0b6d-42c5-8057-f37120ee4b94 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:29.130463 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b591ed-91d7-447e-a82c-db400c7078be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:29.131766 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b591ed-91d7-447e-a82c-db400c7078be None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:29.132222 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b591ed-91d7-447e-a82c-db400c7078be None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:29.132705 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b591ed-91d7-447e-a82c-db400c7078be None admin] PATH_INFO: `/v3/projects/d820cdcb6c0741e380190e6fe127e992` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:29.134641 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b591ed-91d7-447e-a82c-db400c7078be None admin] RBAC: Authorizing `identity:get_project(project_id=d820cdcb6c0741e380190e6fe127e992)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:29.136630 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b591ed-91d7-447e-a82c-db400c7078be None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:29.142171 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 217/450] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jun 16 17:30:28 2022] GET /identity/v3/projects/d820cdcb6c0741e380190e6fe127e992 => generated 302 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:29.146090 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b591ed-91d7-447e-a82c-db400c7078be None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:29.262784 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39274bf8-0b6d-42c5-8057-f37120ee4b94 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:29.278197 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b591ed-91d7-447e-a82c-db400c7078be None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:29.314315 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:29.315320 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b306d7ac-f29c-4997-bb70-406562f8edf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:29.316621 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b306d7ac-f29c-4997-bb70-406562f8edf8 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:29.316934 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b306d7ac-f29c-4997-bb70-406562f8edf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:29.317353 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b306d7ac-f29c-4997-bb70-406562f8edf8 None admin] PATH_INFO: `/v3/projects/f64791acc5674278bd1baac8430d9719` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:29.318636 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b306d7ac-f29c-4997-bb70-406562f8edf8 None admin] RBAC: Authorizing `identity:get_project(project_id=f64791acc5674278bd1baac8430d9719)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:29.320279 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b306d7ac-f29c-4997-bb70-406562f8edf8 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:29.322325 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 234/451] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jun 16 17:30:29 2022] GET /identity/v3/projects/f64791acc5674278bd1baac8430d9719 => generated 344 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:30:29.324182 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd71ee0-f62c-49c8-845c-b6f66323efcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:29.324182 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd71ee0-f62c-49c8-845c-b6f66323efcb None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:29.324182 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd71ee0-f62c-49c8-845c-b6f66323efcb None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:29.324182 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd71ee0-f62c-49c8-845c-b6f66323efcb None admin] PATH_INFO: `/v3/groups/27d023f6aba14910ae7171cb846daf74` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:29.326607 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd71ee0-f62c-49c8-845c-b6f66323efcb None admin] RBAC: Authorizing `identity:get_group(group_id=27d023f6aba14910ae7171cb846daf74)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:29.328898 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd71ee0-f62c-49c8-845c-b6f66323efcb None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:29.331128 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b306d7ac-f29c-4997-bb70-406562f8edf8 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:29.331250 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 218/452] 10.222.0.42 () {56 vars in 1180 bytes} [Thu Jun 16 17:30:29 2022] GET /identity/v3/groups/27d023f6aba14910ae7171cb846daf74 => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:29.336815 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd71ee0-f62c-49c8-845c-b6f66323efcb None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:29.351456 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:29.436861 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b306d7ac-f29c-4997-bb70-406562f8edf8 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:29.457392 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd71ee0-f62c-49c8-845c-b6f66323efcb None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:29.478184 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3be9bc-fd56-4733-8970-f79a4f584698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:29.478929 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3be9bc-fd56-4733-8970-f79a4f584698 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:29.479207 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3be9bc-fd56-4733-8970-f79a4f584698 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:29.479475 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3be9bc-fd56-4733-8970-f79a4f584698 None admin] PATH_INFO: `/v3/groups/2fa79991bc7342d3bd95a309ee21fdb0` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:29.480677 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3be9bc-fd56-4733-8970-f79a4f584698 None admin] RBAC: Authorizing `identity:get_group(group_id=2fa79991bc7342d3bd95a309ee21fdb0)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:29.482099 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3be9bc-fd56-4733-8970-f79a4f584698 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:29.483712 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 235/453] 10.222.0.42 () {56 vars in 1180 bytes} [Thu Jun 16 17:30:29 2022] GET /identity/v3/groups/2fa79991bc7342d3bd95a309ee21fdb0 => generated 229 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:29.489662 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3be9bc-fd56-4733-8970-f79a4f584698 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:29.493043 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0b083d-df6a-49ce-ad89-6ecd4f145559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:29.493704 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0b083d-df6a-49ce-ad89-6ecd4f145559 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:29.494006 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0b083d-df6a-49ce-ad89-6ecd4f145559 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:29.494282 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0b083d-df6a-49ce-ad89-6ecd4f145559 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:29.495111 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0b083d-df6a-49ce-ad89-6ecd4f145559 None admin] RBAC: Adding query filter params (group.id=27d023f6aba14910ae7171cb846daf74, role.id=243fd0a11d94477986deb1dcfb3f31dd, scope.project.id=d820cdcb6c0741e380190e6fe127e992) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:29.495446 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0b083d-df6a-49ce-ad89-6ecd4f145559 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:29.497002 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0b083d-df6a-49ce-ad89-6ecd4f145559 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:29.503663 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 219/454] 10.222.0.42 () {56 vars in 1376 bytes} [Thu Jun 16 17:30:29 2022] GET /identity/v3/role_assignments?group.id=27d023f6aba14910ae7171cb846daf74&scope.project.id=d820cdcb6c0741e380190e6fe127e992&role.id=243fd0a11d94477986deb1dcfb3f31dd => generated 611 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:30:29.582894 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:29.600836 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3be9bc-fd56-4733-8970-f79a4f584698 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:29.640285 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ac1cfb-2b2b-424e-8335-bc20b1c6ec0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:29.641217 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ac1cfb-2b2b-424e-8335-bc20b1c6ec0f None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:29.641504 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ac1cfb-2b2b-424e-8335-bc20b1c6ec0f None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:29.641778 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ac1cfb-2b2b-424e-8335-bc20b1c6ec0f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:29.642623 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ac1cfb-2b2b-424e-8335-bc20b1c6ec0f None admin] RBAC: Adding query filter params (group.id=2fa79991bc7342d3bd95a309ee21fdb0, role.id=3ca2dcbbd29d4a31a3ab78f2299bc239, scope.project.id=f64791acc5674278bd1baac8430d9719) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:29.642940 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ac1cfb-2b2b-424e-8335-bc20b1c6ec0f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:29.644416 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ac1cfb-2b2b-424e-8335-bc20b1c6ec0f None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:29.651064 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 236/455] 10.222.0.42 () {56 vars in 1376 bytes} [Thu Jun 16 17:30:29 2022] GET /identity/v3/role_assignments?group.id=2fa79991bc7342d3bd95a309ee21fdb0&scope.project.id=f64791acc5674278bd1baac8430d9719&role.id=3ca2dcbbd29d4a31a3ab78f2299bc239 => generated 611 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:30:32.469606 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed77b98-b86c-42e0-9cb9-653c8a15ecbc None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:32.470161 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed77b98-b86c-42e0-9cb9-653c8a15ecbc None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:32.470161 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed77b98-b86c-42e0-9cb9-653c8a15ecbc None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:32.471970 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 220/456] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:30:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:30:32.477568 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ac5744-85f8-437c-886e-b9adef670345 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:32.478079 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ac5744-85f8-437c-886e-b9adef670345 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:32.478153 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ac5744-85f8-437c-886e-b9adef670345 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:32.505429 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-05ac5744-85f8-437c-886e-b9adef670345 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:30:32.507060 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-05ac5744-85f8-437c-886e-b9adef670345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:30:32.673265 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 237/457] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:30:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:30:32.673453 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e0e925-a507-4709-899d-cbb092c9a781 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:32.673729 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e0e925-a507-4709-899d-cbb092c9a781 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:32.674011 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e0e925-a507-4709-899d-cbb092c9a781 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:32.675612 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 221/458] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:30:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:30:32.690215 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efecc79-b4bf-4c99-bce7-20837fc62574 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:32.690494 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efecc79-b4bf-4c99-bce7-20837fc62574 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:32.690774 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efecc79-b4bf-4c99-bce7-20837fc62574 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:32.721600 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e0e925-a507-4709-899d-cbb092c9a781 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:32.721790 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-4efecc79-b4bf-4c99-bce7-20837fc62574 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:30:32.722794 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-4efecc79-b4bf-4c99-bce7-20837fc62574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:30:32.814229 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 238/459] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:30:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 17:30:32.815782 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd4a2ba-c752-4dc2-b3d2-2664135991d7 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:32.816079 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd4a2ba-c752-4dc2-b3d2-2664135991d7 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:32.817098 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd4a2ba-c752-4dc2-b3d2-2664135991d7 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:32.818697 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 239/460] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:30:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:30:32.827769 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba435610-95cf-4cad-8de0-2fd0d3e6b8ee None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:32.828973 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba435610-95cf-4cad-8de0-2fd0d3e6b8ee None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:32.829721 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba435610-95cf-4cad-8de0-2fd0d3e6b8ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:32.855289 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-ba435610-95cf-4cad-8de0-2fd0d3e6b8ee None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:30:32.856509 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-ba435610-95cf-4cad-8de0-2fd0d3e6b8ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:30:32.860074 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:32.872752 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:32.923298 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e0e925-a507-4709-899d-cbb092c9a781 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:32.959998 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 240/461] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:30:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:30:32.964866 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba435610-95cf-4cad-8de0-2fd0d3e6b8ee None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:32.986890 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18df4c9-e11b-446a-b137-dab77a9413a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:32.986890 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18df4c9-e11b-446a-b137-dab77a9413a8 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:32.986890 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18df4c9-e11b-446a-b137-dab77a9413a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:32.986890 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18df4c9-e11b-446a-b137-dab77a9413a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:32.999165 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18df4c9-e11b-446a-b137-dab77a9413a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:33.001728 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18df4c9-e11b-446a-b137-dab77a9413a8 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:33.023129 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-d18df4c9-e11b-446a-b137-dab77a9413a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:30:33.023129 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:30:33.023129 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:30:33.023129 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:30:33.023129 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:30:33.023129 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:30:33.023129 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:30:33.023129 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:30:33.023129 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:30:33.023129 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:30:33.023129 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:30:33.023129 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:30:33.023129 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:30:33.023129 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:30:33.023129 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:30:33.024061 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:30:33.024061 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:30:33.024061 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:30:33.024061 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:30:33.024061 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:30:33.024061 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:30:33.024061 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:30:33.024061 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:30:33.024061 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:30:33.024061 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:30:33.024061 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:30:33.024061 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:30:33.024061 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:30:33.024061 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:30:33.024061 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:30:33.024061 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:30:33.024061 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:30:33.024933 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:30:33.024933 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:30:33.024933 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:30:33.024933 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:30:33.027213 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 222/462] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jun 16 17:30:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:30:33.028346 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18df4c9-e11b-446a-b137-dab77a9413a8 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:33.100457 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba435610-95cf-4cad-8de0-2fd0d3e6b8ee None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:33.138408 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d1b7ef-a8b1-47ff-95d3-eaf1a99fb663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:33.139345 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d1b7ef-a8b1-47ff-95d3-eaf1a99fb663 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:33.139622 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d1b7ef-a8b1-47ff-95d3-eaf1a99fb663 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:33.139920 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d1b7ef-a8b1-47ff-95d3-eaf1a99fb663 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:33.147720 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:33.152362 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d1b7ef-a8b1-47ff-95d3-eaf1a99fb663 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:33.152877 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18df4c9-e11b-446a-b137-dab77a9413a8 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:33.153934 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d1b7ef-a8b1-47ff-95d3-eaf1a99fb663 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:33.173841 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:33.177390 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-54d1b7ef-a8b1-47ff-95d3-eaf1a99fb663 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:30:33.177390 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:30:33.177390 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:30:33.177390 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:30:33.177390 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:30:33.177390 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:30:33.177390 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:30:33.177390 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:30:33.177390 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:30:33.177390 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:30:33.177390 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:30:33.177390 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:30:33.177390 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:30:33.177390 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:30:33.177390 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:30:33.178248 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:30:33.178248 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:30:33.178248 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:30:33.178248 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:30:33.178248 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:30:33.178248 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:30:33.178248 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:30:33.178248 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:30:33.178248 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:30:33.178248 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:30:33.178248 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:30:33.178248 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:30:33.178248 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:30:33.178248 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:30:33.178248 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:30:33.178248 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:30:33.178248 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:30:33.179101 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:30:33.179101 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:30:33.179101 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:30:33.179101 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:30:33.180618 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 241/463] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jun 16 17:30:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:30:33.181956 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d1b7ef-a8b1-47ff-95d3-eaf1a99fb663 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:33.211610 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ba4f2e-b6e2-4c49-b956-f36bde6bac2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:33.212425 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ba4f2e-b6e2-4c49-b956-f36bde6bac2f None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:33.212715 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ba4f2e-b6e2-4c49-b956-f36bde6bac2f None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:33.212990 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ba4f2e-b6e2-4c49-b956-f36bde6bac2f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:33.230650 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ba4f2e-b6e2-4c49-b956-f36bde6bac2f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:33.232152 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ba4f2e-b6e2-4c49-b956-f36bde6bac2f None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:33.252067 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-25ba4f2e-b6e2-4c49-b956-f36bde6bac2f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:30:33.252067 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:30:33.252067 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:30:33.252067 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:30:33.252067 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:30:33.252067 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:30:33.252067 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:30:33.252067 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:30:33.252067 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:30:33.252067 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:30:33.252067 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:30:33.252067 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:30:33.252067 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:30:33.252067 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:30:33.252067 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:30:33.252981 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:30:33.252981 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:30:33.252981 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:30:33.252981 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:30:33.252981 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:30:33.252981 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:30:33.252981 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:30:33.252981 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:30:33.252981 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:30:33.252981 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:30:33.252981 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:30:33.252981 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:30:33.252981 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:30:33.252981 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:30:33.252981 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:30:33.252981 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:30:33.252981 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:30:33.254710 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:30:33.254710 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:30:33.254710 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:30:33.254710 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:30:33.254710 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 223/464] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jun 16 17:30:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:30:33.254710 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24096c38-242c-4eab-98aa-72972268bd33 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:33.255036 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24096c38-242c-4eab-98aa-72972268bd33 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:33.255036 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24096c38-242c-4eab-98aa-72972268bd33 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:33.260196 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 224/465] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:30:33 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:30:33.261019 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24096c38-242c-4eab-98aa-72972268bd33 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:33.316271 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d1b7ef-a8b1-47ff-95d3-eaf1a99fb663 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:33.355983 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d356d1-0975-4ae3-b54f-86998de21d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:33.356816 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d356d1-0975-4ae3-b54f-86998de21d1b None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:33.357098 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d356d1-0975-4ae3-b54f-86998de21d1b None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:33.357371 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d356d1-0975-4ae3-b54f-86998de21d1b None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:33.358092 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d356d1-0975-4ae3-b54f-86998de21d1b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:33.358398 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d356d1-0975-4ae3-b54f-86998de21d1b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:33.360595 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d356d1-0975-4ae3-b54f-86998de21d1b None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:33.382906 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 242/466] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jun 16 17:30:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:33.382906 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d356d1-0975-4ae3-b54f-86998de21d1b None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:33.387091 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24096c38-242c-4eab-98aa-72972268bd33 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:33.430275 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe59801c-487a-45f3-88c7-f99f393393e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:33.431268 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe59801c-487a-45f3-88c7-f99f393393e8 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:33.431722 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe59801c-487a-45f3-88c7-f99f393393e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:33.432166 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe59801c-487a-45f3-88c7-f99f393393e8 None admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:33.433086 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe59801c-487a-45f3-88c7-f99f393393e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:33.433580 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe59801c-487a-45f3-88c7-f99f393393e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:33.435088 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe59801c-487a-45f3-88c7-f99f393393e8 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:33.436686 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:33.447694 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 225/467] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jun 16 17:30:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:33.449066 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a319984-e3c0-4cab-9fc9-133f1b108606 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:33.449356 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a319984-e3c0-4cab-9fc9-133f1b108606 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:33.449632 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a319984-e3c0-4cab-9fc9-133f1b108606 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:33.470643 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-5a319984-e3c0-4cab-9fc9-133f1b108606 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:30:33.472513 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-5a319984-e3c0-4cab-9fc9-133f1b108606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:30:33.494833 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:33.505559 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d356d1-0975-4ae3-b54f-86998de21d1b None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:33.543782 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b4272a-a289-4ca1-824d-08061e407a49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:33.544478 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b4272a-a289-4ca1-824d-08061e407a49 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:33.544766 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b4272a-a289-4ca1-824d-08061e407a49 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:33.545094 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b4272a-a289-4ca1-824d-08061e407a49 None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:33.545820 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b4272a-a289-4ca1-824d-08061e407a49 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:33.546125 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b4272a-a289-4ca1-824d-08061e407a49 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:33.547828 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b4272a-a289-4ca1-824d-08061e407a49 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:33.559647 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 226/468] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:30:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:30:33.561116 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a319984-e3c0-4cab-9fc9-133f1b108606 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:33.563058 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 243/469] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jun 16 17:30:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:33.564359 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b4272a-a289-4ca1-824d-08061e407a49 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:33.691773 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b4272a-a289-4ca1-824d-08061e407a49 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:33.698120 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a319984-e3c0-4cab-9fc9-133f1b108606 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:33.704272 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:33.729897 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70cac7b-648d-46b8-931c-d982b7e32560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:33.730580 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70cac7b-648d-46b8-931c-d982b7e32560 None admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:33.730865 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70cac7b-648d-46b8-931c-d982b7e32560 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:33.731145 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70cac7b-648d-46b8-931c-d982b7e32560 None admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:33.731860 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70cac7b-648d-46b8-931c-d982b7e32560 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:33.733340 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70cac7b-648d-46b8-931c-d982b7e32560 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:33.737950 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee724f0e-ef7d-4b44-8d8f-df2b22157ae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:33.738617 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee724f0e-ef7d-4b44-8d8f-df2b22157ae5 None admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:33.738899 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee724f0e-ef7d-4b44-8d8f-df2b22157ae5 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:33.739181 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee724f0e-ef7d-4b44-8d8f-df2b22157ae5 None admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:33.739877 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee724f0e-ef7d-4b44-8d8f-df2b22157ae5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:33.741349 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee724f0e-ef7d-4b44-8d8f-df2b22157ae5 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:33.753951 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 244/470] 10.222.0.42 () {60 vars in 1132 bytes} [Thu Jun 16 17:30:33 2022] POST /identity/v3/users => generated 248 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:30:33.754740 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70cac7b-648d-46b8-931c-d982b7e32560 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:33.784508 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 227/471] 10.222.0.42 () {60 vars in 1132 bytes} [Thu Jun 16 17:30:33 2022] POST /identity/v3/users => generated 246 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:30:33.784508 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee724f0e-ef7d-4b44-8d8f-df2b22157ae5 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:33.804605 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:33.882668 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70cac7b-648d-46b8-931c-d982b7e32560 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:33.927004 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e48c404-f70e-479d-a45f-16d7e21b0b60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:33.927971 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e48c404-f70e-479d-a45f-16d7e21b0b60 None admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:33.928254 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e48c404-f70e-479d-a45f-16d7e21b0b60 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:33.928588 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e48c404-f70e-479d-a45f-16d7e21b0b60 None admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:33.932403 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e48c404-f70e-479d-a45f-16d7e21b0b60 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:33.932403 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e48c404-f70e-479d-a45f-16d7e21b0b60 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:33.938381 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee724f0e-ef7d-4b44-8d8f-df2b22157ae5 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:33.959716 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 245/472] 10.222.0.42 () {60 vars in 1132 bytes} [Thu Jun 16 17:30:33 2022] POST /identity/v3/users => generated 248 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:30:33.961924 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d20225-7c7d-41f9-b899-8a5aa9f75e7e None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:33.962976 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d20225-7c7d-41f9-b899-8a5aa9f75e7e None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:33.963275 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d20225-7c7d-41f9-b899-8a5aa9f75e7e None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:33.964832 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 246/473] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:30:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:30:33.970751 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaf78cc-12cb-4487-9423-a9be4cc97bf0 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:33.971056 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaf78cc-12cb-4487-9423-a9be4cc97bf0 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:33.972567 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaf78cc-12cb-4487-9423-a9be4cc97bf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:33.995077 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-cbaf78cc-12cb-4487-9423-a9be4cc97bf0 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:30:33.997139 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-cbaf78cc-12cb-4487-9423-a9be4cc97bf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:30:33.997837 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bf0976-419f-407b-bade-ffad1ed7bb9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:33.999534 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bf0976-419f-407b-bade-ffad1ed7bb9f None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:33.999534 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bf0976-419f-407b-bade-ffad1ed7bb9f None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:33.999534 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bf0976-419f-407b-bade-ffad1ed7bb9f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:34.017929 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bf0976-419f-407b-bade-ffad1ed7bb9f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:34.020769 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bf0976-419f-407b-bade-ffad1ed7bb9f None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:34.030662 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:34.051915 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-f5bf0976-419f-407b-bade-ffad1ed7bb9f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:30:34.051915 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:30:34.051915 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:30:34.051915 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:30:34.051915 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:30:34.051915 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:30:34.051915 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:30:34.051915 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:30:34.051915 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:30:34.051915 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:30:34.051915 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:30:34.051915 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:30:34.051915 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:30:34.051915 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:30:34.051915 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:30:34.052849 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:30:34.052849 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:30:34.052849 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:30:34.052849 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:30:34.052849 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:30:34.052849 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:30:34.052849 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:30:34.052849 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:30:34.052849 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:30:34.052849 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:30:34.052849 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:30:34.052849 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:30:34.052849 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:30:34.052849 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:30:34.052849 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:30:34.052849 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:30:34.052849 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:30:34.053704 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:30:34.053704 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:30:34.053704 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:30:34.053704 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:30:34.056203 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 228/474] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jun 16 17:30:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:30:34.063941 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bf0976-419f-407b-bade-ffad1ed7bb9f None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:34.126290 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 247/475] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:30:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:30:34.155485 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaf78cc-12cb-4487-9423-a9be4cc97bf0 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:34.167497 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:34.231271 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bf0976-419f-407b-bade-ffad1ed7bb9f None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:34.285897 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb02abe-d4db-426c-9531-5d8feca9daef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:34.287012 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb02abe-d4db-426c-9531-5d8feca9daef None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:34.287317 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb02abe-d4db-426c-9531-5d8feca9daef None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:34.287599 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb02abe-d4db-426c-9531-5d8feca9daef None admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:34.288338 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb02abe-d4db-426c-9531-5d8feca9daef None admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:34.288694 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb02abe-d4db-426c-9531-5d8feca9daef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:34.290078 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb02abe-d4db-426c-9531-5d8feca9daef None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:34.305573 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaf78cc-12cb-4487-9423-a9be4cc97bf0 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:34.307751 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 229/476] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jun 16 17:30:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:34.313149 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb02abe-d4db-426c-9531-5d8feca9daef None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:34.344580 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b00137-0ff4-4f20-95c6-6eec3c8c0d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:34.345660 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b00137-0ff4-4f20-95c6-6eec3c8c0d0d None admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:34.346136 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b00137-0ff4-4f20-95c6-6eec3c8c0d0d None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:34.346621 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b00137-0ff4-4f20-95c6-6eec3c8c0d0d None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:34.347552 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b00137-0ff4-4f20-95c6-6eec3c8c0d0d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:34.349341 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b00137-0ff4-4f20-95c6-6eec3c8c0d0d None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:34.356488 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.sql.core [None req-88b00137-0ff4-4f20-95c6-6eec3c8c0d0d None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 16 17:30:34.356488 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 16 17:30:34.356488 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [parameters: {'id': '8a781cd09d57491ebc358e7e55dee51b', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 16 17:30:34.356488 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80358) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 16 17:30:34.381315 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-88b00137-0ff4-4f20-95c6-6eec3c8c0d0d None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 16 17:30:34.381315 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:30:34.381315 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 16 17:30:34.381315 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 16 17:30:34.381315 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 16 17:30:34.381315 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 16 17:30:34.381315 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 16 17:30:34.381315 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application result = self._query(query) Jun 16 17:30:34.381315 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 16 17:30:34.381315 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application conn.query(q) Jun 16 17:30:34.381315 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 16 17:30:34.381315 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 16 17:30:34.381315 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 16 17:30:34.381315 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application result.read() Jun 16 17:30:34.381315 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 16 17:30:34.382277 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 16 17:30:34.382277 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 16 17:30:34.382277 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 16 17:30:34.382277 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 16 17:30:34.382277 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 16 17:30:34.382277 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 16 17:30:34.382277 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 16 17:30:34.382277 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 16 17:30:34.382277 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:30:34.382277 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 16 17:30:34.382277 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:30:34.382277 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:30:34.382277 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 16 17:30:34.382277 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 16 17:30:34.382277 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jun 16 17:30:34.382277 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return ref.to_dict() Jun 16 17:30:34.382277 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 16 17:30:34.382277 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application next(self.gen) Jun 16 17:30:34.383178 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 16 17:30:34.383178 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application yield resource Jun 16 17:30:34.383178 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 16 17:30:34.383178 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application next(self.gen) Jun 16 17:30:34.383178 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 16 17:30:34.383178 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self.session.rollback() Jun 16 17:30:34.383178 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 16 17:30:34.383178 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self.force_reraise() Jun 16 17:30:34.383178 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 16 17:30:34.383178 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise self.value Jun 16 17:30:34.383178 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 16 17:30:34.383178 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 16 17:30:34.383178 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 16 17:30:34.383178 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application session.commit() Jun 16 17:30:34.383178 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 16 17:30:34.383178 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 16 17:30:34.383178 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 16 17:30:34.383999 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self._prepare_impl() Jun 16 17:30:34.383999 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 16 17:30:34.383999 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self.session.flush() Jun 16 17:30:34.383999 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 16 17:30:34.383999 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self._flush(objects) Jun 16 17:30:34.383999 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 16 17:30:34.383999 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 16 17:30:34.383999 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 16 17:30:34.383999 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application compat.raise_( Jun 16 17:30:34.383999 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 16 17:30:34.383999 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception Jun 16 17:30:34.383999 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 16 17:30:34.383999 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application flush_context.execute() Jun 16 17:30:34.383999 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 16 17:30:34.383999 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rec.execute(self) Jun 16 17:30:34.383999 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 16 17:30:34.383999 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 16 17:30:34.384942 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 16 17:30:34.384942 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 16 17:30:34.384942 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 16 17:30:34.384942 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 16 17:30:34.384942 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 16 17:30:34.384942 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 16 17:30:34.384942 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 16 17:30:34.384942 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 16 17:30:34.384942 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 16 17:30:34.384942 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 16 17:30:34.384942 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 16 17:30:34.384942 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 16 17:30:34.384942 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 16 17:30:34.384942 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 16 17:30:34.384942 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 16 17:30:34.384942 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception Jun 16 17:30:34.385755 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 16 17:30:34.385755 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 16 17:30:34.385755 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 16 17:30:34.385755 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 16 17:30:34.385755 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 16 17:30:34.385755 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application result = self._query(query) Jun 16 17:30:34.385755 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 16 17:30:34.385755 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application conn.query(q) Jun 16 17:30:34.385755 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 16 17:30:34.385755 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 16 17:30:34.385755 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 16 17:30:34.385755 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application result.read() Jun 16 17:30:34.385755 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 16 17:30:34.385755 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 16 17:30:34.385755 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 16 17:30:34.385755 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 16 17:30:34.386519 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 16 17:30:34.386519 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 16 17:30:34.386519 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 16 17:30:34.386519 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 16 17:30:34.386519 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 16 17:30:34.386519 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 16 17:30:34.386519 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [parameters: {'id': '8a781cd09d57491ebc358e7e55dee51b', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 16 17:30:34.386519 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 16 17:30:34.386519 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:30:34.386519 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:30:34.386519 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:30:34.386519 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:30:34.386519 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:30:34.386519 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:30:34.386519 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:30:34.386519 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:30:34.387297 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:30:34.387297 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:30:34.387297 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:30:34.387297 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:30:34.387297 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:30:34.387297 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:30:34.387297 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jun 16 17:30:34.387297 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jun 16 17:30:34.387297 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:30:34.387297 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:30:34.387297 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jun 16 17:30:34.387297 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jun 16 17:30:34.387297 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 16 17:30:34.387297 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 16 17:30:34.387297 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 16 17:30:34.387297 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:30:34.392545 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 248/477] 10.222.0.42 () {60 vars in 1131 bytes} [Thu Jun 16 17:30:34 2022] POST /identity/v3/roles => generated 144 bytes in 237 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:30:34.398807 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b00137-0ff4-4f20-95c6-6eec3c8c0d0d None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:34.420655 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:34.430672 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb02abe-d4db-426c-9531-5d8feca9daef None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:34.459057 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:34.473377 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a42bc2-a840-4c5c-9644-00d4fa6e4825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:34.474069 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a42bc2-a840-4c5c-9644-00d4fa6e4825 None admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:34.474298 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a42bc2-a840-4c5c-9644-00d4fa6e4825 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:34.474580 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a42bc2-a840-4c5c-9644-00d4fa6e4825 None admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:34.475299 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a42bc2-a840-4c5c-9644-00d4fa6e4825 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:34.476806 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a42bc2-a840-4c5c-9644-00d4fa6e4825 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:34.503739 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 230/478] 10.222.0.42 () {60 vars in 1132 bytes} [Thu Jun 16 17:30:34 2022] POST /identity/v3/users => generated 249 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:30:34.544678 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b00137-0ff4-4f20-95c6-6eec3c8c0d0d None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:34.616563 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1c3e48-97c3-41e4-ab4b-6e24eec33c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:34.617776 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1c3e48-97c3-41e4-ab4b-6e24eec33c5b None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:34.618270 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1c3e48-97c3-41e4-ab4b-6e24eec33c5b None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:34.618737 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1c3e48-97c3-41e4-ab4b-6e24eec33c5b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:34.645626 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1c3e48-97c3-41e4-ab4b-6e24eec33c5b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:30:34.654041 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-0c1c3e48-97c3-41e4-ab4b-6e24eec33c5b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 17:30:34.654041 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:30:34.654041 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:30:34.654041 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:30:34.654041 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:30:34.654041 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:30:34.654041 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:30:34.654041 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:30:34.654041 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:30:34.654041 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:30:34.654041 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:30:34.654041 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:30:34.654041 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:30:34.654041 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:30:34.654041 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:30:34.654041 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:30:34.655032 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:30:34.655032 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:30:34.655032 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:30:34.655032 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:30:34.655032 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:30:34.655032 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:30:34.655032 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:30:34.655032 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:30:34.655032 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:30:34.655032 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:30:34.655032 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:30:34.655032 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:30:34.655032 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:30:34.655032 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:30:34.655032 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:30:34.655032 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:30:34.655834 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:30:34.655834 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:30:34.655834 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:30:34.655834 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:30:34.655834 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:30:34.655834 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:30:34.655834 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:30:34.655834 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:30:34.655834 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:30:34.655834 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:30:34.655834 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 17:30:34.655834 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:30:34.655834 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 249/479] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Jun 16 17:30:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:30:34.663578 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a42bc2-a840-4c5c-9644-00d4fa6e4825 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:34.773644 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a42bc2-a840-4c5c-9644-00d4fa6e4825 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:34.811671 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe24247-d3a0-49d7-8db8-62168e301fa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:34.813016 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe24247-d3a0-49d7-8db8-62168e301fa6 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:34.813324 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe24247-d3a0-49d7-8db8-62168e301fa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:34.813603 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe24247-d3a0-49d7-8db8-62168e301fa6 None admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:34.814419 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe24247-d3a0-49d7-8db8-62168e301fa6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:34.814727 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe24247-d3a0-49d7-8db8-62168e301fa6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:34.816117 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe24247-d3a0-49d7-8db8-62168e301fa6 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:34.825414 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 231/480] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jun 16 17:30:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:36.656428 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238591d6-9268-4acb-b263-0f7cec4cee28 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:36.656907 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238591d6-9268-4acb-b263-0f7cec4cee28 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:36.656907 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238591d6-9268-4acb-b263-0f7cec4cee28 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:36.658512 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 250/481] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:30:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:30:36.664493 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aaaf8d-7b13-4e20-a6ee-4f40b699006c None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:36.664789 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aaaf8d-7b13-4e20-a6ee-4f40b699006c None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:36.665086 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aaaf8d-7b13-4e20-a6ee-4f40b699006c None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:36.666391 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 251/482] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:30:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:30:36.667889 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870ceb39-84f8-4a1a-a691-b1f69c91d15a None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:36.668306 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870ceb39-84f8-4a1a-a691-b1f69c91d15a None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:36.668499 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870ceb39-84f8-4a1a-a691-b1f69c91d15a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:36.672914 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49ccc86-ba87-417f-b49c-1c4d3c6c0bd4 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:36.673201 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49ccc86-ba87-417f-b49c-1c4d3c6c0bd4 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:36.673482 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49ccc86-ba87-417f-b49c-1c4d3c6c0bd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:36.702863 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-c49ccc86-ba87-417f-b49c-1c4d3c6c0bd4 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:30:36.704989 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-c49ccc86-ba87-417f-b49c-1c4d3c6c0bd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:30:36.707733 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-870ceb39-84f8-4a1a-a691-b1f69c91d15a None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:30:36.709165 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-870ceb39-84f8-4a1a-a691-b1f69c91d15a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:30:36.789858 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 252/483] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:30:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 17:30:36.814888 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:36.840248 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49ccc86-ba87-417f-b49c-1c4d3c6c0bd4 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:36.858477 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:36.925060 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 232/484] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:30:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:30:36.998876 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870ceb39-84f8-4a1a-a691-b1f69c91d15a None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:37.024461 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49ccc86-ba87-417f-b49c-1c4d3c6c0bd4 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:37.081821 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9aa7e8a-f062-433b-a247-e6e410ef2e02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:37.088396 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aa7e8a-f062-433b-a247-e6e410ef2e02 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:37.088396 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aa7e8a-f062-433b-a247-e6e410ef2e02 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:37.088396 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aa7e8a-f062-433b-a247-e6e410ef2e02 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:37.106309 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9aa7e8a-f062-433b-a247-e6e410ef2e02 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:30:37.119570 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-b9aa7e8a-f062-433b-a247-e6e410ef2e02 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:30:37.119570 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:30:37.119570 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:30:37.119570 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:30:37.119570 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:30:37.119570 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:30:37.119570 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:30:37.119570 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:30:37.119570 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:30:37.119570 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:30:37.119570 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:30:37.119570 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:30:37.119570 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:30:37.119570 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:30:37.119570 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:30:37.119570 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:30:37.120535 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:30:37.120535 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:30:37.120535 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:30:37.120535 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:30:37.120535 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:30:37.120535 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:30:37.120535 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:30:37.120535 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:30:37.120535 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:30:37.120535 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:30:37.120535 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:30:37.120535 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:30:37.120535 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:30:37.120535 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:30:37.120535 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:30:37.120535 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:30:37.121310 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:30:37.121310 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:30:37.121310 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:30:37.121310 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:30:37.121310 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:30:37.121310 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:30:37.121310 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:30:37.121310 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:30:37.121310 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:30:37.121310 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:30:37.121310 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:30:37.121310 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:30:37.124780 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 253/485] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Jun 16 17:30:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:30:37.128396 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ff3fe8-5207-497c-b140-b07d3a378420 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:37.128396 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ff3fe8-5207-497c-b140-b07d3a378420 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:37.128396 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ff3fe8-5207-497c-b140-b07d3a378420 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:37.134600 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 254/486] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:30:37 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:30:37.140402 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ff3fe8-5207-497c-b140-b07d3a378420 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:37.207295 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870ceb39-84f8-4a1a-a691-b1f69c91d15a None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:37.288398 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850a9092-7cc9-4287-a163-da4bef3b382e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:37.288398 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850a9092-7cc9-4287-a163-da4bef3b382e None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:37.288398 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850a9092-7cc9-4287-a163-da4bef3b382e None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:37.288398 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850a9092-7cc9-4287-a163-da4bef3b382e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:37.292720 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:37.313062 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850a9092-7cc9-4287-a163-da4bef3b382e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:30:37.324864 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-850a9092-7cc9-4287-a163-da4bef3b382e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:30:37.324864 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:30:37.324864 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:30:37.324864 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:30:37.324864 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:30:37.324864 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:30:37.324864 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:30:37.324864 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:30:37.324864 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:30:37.324864 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:30:37.324864 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:30:37.324864 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:30:37.324864 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:30:37.324864 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:30:37.324864 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:30:37.324864 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:30:37.325865 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:30:37.325865 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:30:37.325865 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:30:37.325865 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:30:37.325865 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:30:37.325865 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:30:37.325865 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:30:37.325865 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:30:37.325865 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:30:37.325865 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:30:37.325865 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:30:37.325865 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:30:37.325865 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:30:37.325865 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:30:37.325865 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:30:37.325865 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:30:37.326671 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:30:37.326671 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:30:37.326671 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:30:37.326671 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:30:37.326671 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:30:37.326671 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:30:37.326671 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:30:37.326671 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:30:37.326671 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:30:37.326671 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:30:37.326671 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:30:37.326671 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:30:37.333057 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 233/487] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Jun 16 17:30:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:30:37.335912 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c567346e-2bc8-4f55-8f73-3eefa76aa74d None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:37.338060 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c567346e-2bc8-4f55-8f73-3eefa76aa74d None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:37.338060 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c567346e-2bc8-4f55-8f73-3eefa76aa74d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:37.363914 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-c567346e-2bc8-4f55-8f73-3eefa76aa74d None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:30:37.365200 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-c567346e-2bc8-4f55-8f73-3eefa76aa74d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:30:37.388597 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ff3fe8-5207-497c-b140-b07d3a378420 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:37.399250 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:37.452183 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 234/488] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:30:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:30:37.454209 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c567346e-2bc8-4f55-8f73-3eefa76aa74d None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:37.461408 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db050ed-d05c-49d3-983f-ab83d6fe6795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:37.463210 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db050ed-d05c-49d3-983f-ab83d6fe6795 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:37.463496 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db050ed-d05c-49d3-983f-ab83d6fe6795 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:37.463774 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db050ed-d05c-49d3-983f-ab83d6fe6795 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:37.466505 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db050ed-d05c-49d3-983f-ab83d6fe6795 None admin] RBAC: Adding query filter params (name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:37.466829 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db050ed-d05c-49d3-983f-ab83d6fe6795 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:37.472585 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db050ed-d05c-49d3-983f-ab83d6fe6795 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:37.490479 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 255/489] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Jun 16 17:30:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:37.491280 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db050ed-d05c-49d3-983f-ab83d6fe6795 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:37.601974 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c567346e-2bc8-4f55-8f73-3eefa76aa74d None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:37.640078 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db050ed-d05c-49d3-983f-ab83d6fe6795 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:37.659567 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450cb27a-74e4-4a86-9fd5-f8581d2b9bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:37.660236 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450cb27a-74e4-4a86-9fd5-f8581d2b9bdb None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:37.660554 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450cb27a-74e4-4a86-9fd5-f8581d2b9bdb None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:37.660833 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450cb27a-74e4-4a86-9fd5-f8581d2b9bdb None admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:37.661607 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450cb27a-74e4-4a86-9fd5-f8581d2b9bdb None admin] RBAC: Adding query filter params (name=service) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:37.661913 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450cb27a-74e4-4a86-9fd5-f8581d2b9bdb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:37.663426 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450cb27a-74e4-4a86-9fd5-f8581d2b9bdb None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:37.680695 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 235/490] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Jun 16 17:30:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:37.681697 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450cb27a-74e4-4a86-9fd5-f8581d2b9bdb None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:37.686646 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:37.697263 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237adaaa-dd47-45a4-8ce2-33d50f62e97c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:37.699429 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237adaaa-dd47-45a4-8ce2-33d50f62e97c None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:37.699914 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237adaaa-dd47-45a4-8ce2-33d50f62e97c None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:37.700606 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237adaaa-dd47-45a4-8ce2-33d50f62e97c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:37.714453 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237adaaa-dd47-45a4-8ce2-33d50f62e97c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:30:37.716084 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:37.724555 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-237adaaa-dd47-45a4-8ce2-33d50f62e97c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:30:37.724555 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:30:37.724555 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:30:37.724555 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:30:37.724555 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:30:37.724555 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:30:37.724555 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:30:37.724555 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:30:37.724555 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:30:37.724555 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:30:37.724555 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:30:37.724555 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:30:37.724555 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:30:37.724555 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:30:37.724555 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:30:37.724555 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:30:37.725505 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:30:37.725505 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:30:37.725505 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:30:37.725505 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:30:37.725505 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:30:37.725505 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:30:37.725505 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:30:37.725505 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:30:37.725505 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:30:37.725505 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:30:37.725505 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:30:37.725505 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:30:37.725505 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:30:37.725505 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:30:37.725505 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:30:37.725505 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:30:37.726303 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:30:37.726303 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:30:37.726303 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:30:37.726303 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:30:37.726303 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:30:37.726303 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:30:37.726303 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:30:37.726303 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:30:37.726303 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:30:37.726303 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:30:37.726303 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:30:37.726303 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:30:37.730827 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 256/491] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Jun 16 17:30:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:30:37.731837 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237adaaa-dd47-45a4-8ce2-33d50f62e97c None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:37.826352 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450cb27a-74e4-4a86-9fd5-f8581d2b9bdb None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:37.848942 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237adaaa-dd47-45a4-8ce2-33d50f62e97c None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:37.866270 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdf3784-3732-42fe-97bf-e8ab50a4a471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:37.867552 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdf3784-3732-42fe-97bf-e8ab50a4a471 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:37.868028 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdf3784-3732-42fe-97bf-e8ab50a4a471 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:37.868310 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdf3784-3732-42fe-97bf-e8ab50a4a471 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:37.879922 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdf3784-3732-42fe-97bf-e8ab50a4a471 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:37.881436 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdf3784-3732-42fe-97bf-e8ab50a4a471 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:37.893137 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b12243-5700-40d8-8b04-235fc06b4242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:37.893943 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b12243-5700-40d8-8b04-235fc06b4242 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:37.894223 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b12243-5700-40d8-8b04-235fc06b4242 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:37.894496 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b12243-5700-40d8-8b04-235fc06b4242 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:37.900974 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-8fdf3784-3732-42fe-97bf-e8ab50a4a471 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:30:37.900974 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:30:37.900974 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:30:37.900974 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:30:37.900974 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:30:37.900974 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:30:37.900974 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:30:37.900974 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:30:37.900974 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:30:37.900974 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:30:37.900974 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:30:37.900974 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:30:37.900974 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:30:37.900974 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:30:37.900974 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:30:37.901876 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:30:37.901876 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:30:37.901876 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:30:37.901876 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:30:37.901876 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:30:37.901876 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:30:37.901876 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:30:37.901876 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:30:37.901876 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:30:37.901876 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:30:37.901876 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:30:37.901876 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:30:37.901876 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:30:37.901876 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:30:37.901876 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:30:37.901876 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:30:37.901876 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:30:37.903108 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:30:37.903108 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:30:37.903108 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:30:37.903108 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:30:37.903108 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 236/492] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jun 16 17:30:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:30:37.904064 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bad1ba-6e03-4ea0-acc7-78d139640365 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:37.904346 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bad1ba-6e03-4ea0-acc7-78d139640365 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:37.904645 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bad1ba-6e03-4ea0-acc7-78d139640365 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:37.906177 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 237/493] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:30:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:30:37.906307 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b12243-5700-40d8-8b04-235fc06b4242 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:37.909656 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b12243-5700-40d8-8b04-235fc06b4242 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:37.909773 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bad1ba-6e03-4ea0-acc7-78d139640365 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:37.929751 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-27b12243-5700-40d8-8b04-235fc06b4242 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:30:37.929751 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:30:37.929751 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:30:37.929751 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:30:37.929751 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:30:37.929751 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:30:37.929751 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:30:37.929751 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:30:37.929751 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:30:37.929751 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:30:37.929751 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:30:37.929751 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:30:37.929751 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:30:37.929751 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:30:37.929751 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:30:37.930675 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:30:37.930675 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:30:37.930675 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:30:37.930675 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:30:37.930675 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:30:37.930675 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:30:37.930675 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:30:37.930675 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:30:37.930675 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:30:37.930675 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:30:37.930675 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:30:37.930675 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:30:37.930675 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:30:37.930675 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:30:37.930675 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:30:37.930675 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:30:37.930675 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:30:37.931515 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:30:37.931515 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:30:37.931515 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:30:37.931515 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:30:37.931515 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 257/494] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jun 16 17:30:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:30:37.932927 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ca5f2b-9e86-4fed-b0b0-5205d0a064cb None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:37.933217 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ca5f2b-9e86-4fed-b0b0-5205d0a064cb None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:37.933602 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ca5f2b-9e86-4fed-b0b0-5205d0a064cb None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:37.940248 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 258/495] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:30:37 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:30:37.941088 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ca5f2b-9e86-4fed-b0b0-5205d0a064cb None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:37.995437 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:38.004079 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:38.031806 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bad1ba-6e03-4ea0-acc7-78d139640365 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:38.066114 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ca5f2b-9e86-4fed-b0b0-5205d0a064cb None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:38.072964 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9912a731-0f81-43f3-b331-44186235a457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:38.074061 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9912a731-0f81-43f3-b331-44186235a457 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:38.074355 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9912a731-0f81-43f3-b331-44186235a457 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:38.074629 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9912a731-0f81-43f3-b331-44186235a457 None admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:38.075377 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9912a731-0f81-43f3-b331-44186235a457 None admin] RBAC: Adding query filter params (name=service) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:38.075682 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9912a731-0f81-43f3-b331-44186235a457 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:38.077101 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9912a731-0f81-43f3-b331-44186235a457 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:38.086781 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 238/496] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Jun 16 17:30:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:38.088057 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c037f453-8df4-43bc-b535-326141557322 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:38.088343 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c037f453-8df4-43bc-b535-326141557322 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:38.088648 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c037f453-8df4-43bc-b535-326141557322 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:38.106113 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-c037f453-8df4-43bc-b535-326141557322 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:30:38.107321 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-c037f453-8df4-43bc-b535-326141557322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:30:38.110335 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7459ef6-e0e7-4308-8df5-15f51c9fb744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:38.111742 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7459ef6-e0e7-4308-8df5-15f51c9fb744 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:38.111742 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7459ef6-e0e7-4308-8df5-15f51c9fb744 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:38.111742 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7459ef6-e0e7-4308-8df5-15f51c9fb744 None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:38.112224 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7459ef6-e0e7-4308-8df5-15f51c9fb744 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:38.112570 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7459ef6-e0e7-4308-8df5-15f51c9fb744 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:38.113952 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7459ef6-e0e7-4308-8df5-15f51c9fb744 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:38.127171 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 259/497] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jun 16 17:30:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:38.127855 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7459ef6-e0e7-4308-8df5-15f51c9fb744 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:38.188135 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 239/498] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:30:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:30:38.189612 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405d2853-e562-4aea-a27d-536e3985caa4 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:38.189903 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405d2853-e562-4aea-a27d-536e3985caa4 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:38.190182 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405d2853-e562-4aea-a27d-536e3985caa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:38.208037 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-405d2853-e562-4aea-a27d-536e3985caa4 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:30:38.209255 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-405d2853-e562-4aea-a27d-536e3985caa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:30:38.245273 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:38.249024 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7459ef6-e0e7-4308-8df5-15f51c9fb744 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:38.278057 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:38.295335 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 240/499] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:30:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:30:38.295335 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405d2853-e562-4aea-a27d-536e3985caa4 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:38.295617 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb707d5d-3f3e-4090-9b1b-a269ce3fdad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:38.295617 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb707d5d-3f3e-4090-9b1b-a269ce3fdad2 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:38.295617 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb707d5d-3f3e-4090-9b1b-a269ce3fdad2 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:38.295617 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb707d5d-3f3e-4090-9b1b-a269ce3fdad2 None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:38.295888 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb707d5d-3f3e-4090-9b1b-a269ce3fdad2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:38.295888 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb707d5d-3f3e-4090-9b1b-a269ce3fdad2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:38.295888 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb707d5d-3f3e-4090-9b1b-a269ce3fdad2 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:38.309265 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 260/500] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jun 16 17:30:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:38.310553 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb707d5d-3f3e-4090-9b1b-a269ce3fdad2 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:38.412458 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405d2853-e562-4aea-a27d-536e3985caa4 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:38.425476 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb707d5d-3f3e-4090-9b1b-a269ce3fdad2 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:38.454783 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8812da5c-0b78-43f7-ae47-4a38808a9634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:38.455676 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8812da5c-0b78-43f7-ae47-4a38808a9634 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:38.456033 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8812da5c-0b78-43f7-ae47-4a38808a9634 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:38.456445 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8812da5c-0b78-43f7-ae47-4a38808a9634 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:38.462022 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967b3eda-430d-41d9-b942-46da90b9aaee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:38.462964 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967b3eda-430d-41d9-b942-46da90b9aaee None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:38.463257 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967b3eda-430d-41d9-b942-46da90b9aaee None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:38.463530 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967b3eda-430d-41d9-b942-46da90b9aaee None admin] PATH_INFO: `/v3/users/glance` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:38.467840 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8812da5c-0b78-43f7-ae47-4a38808a9634 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:38.470601 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8812da5c-0b78-43f7-ae47-4a38808a9634 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:38.472306 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967b3eda-430d-41d9-b942-46da90b9aaee None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:38.473956 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967b3eda-430d-41d9-b942-46da90b9aaee None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:38.490316 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-967b3eda-430d-41d9-b942-46da90b9aaee None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 16 17:30:38.490316 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:30:38.490316 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:30:38.490316 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:30:38.490316 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:30:38.490316 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:30:38.490316 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:30:38.490316 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:30:38.490316 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:30:38.490316 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:30:38.490316 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:30:38.490316 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:30:38.490316 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:30:38.490316 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:30:38.490316 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:30:38.490316 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:30:38.491400 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:30:38.491400 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:30:38.491400 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:30:38.491400 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:30:38.491400 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:30:38.491400 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:30:38.491400 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:30:38.491400 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:30:38.491400 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:30:38.491400 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:30:38.491400 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:30:38.491400 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:30:38.491400 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:30:38.491400 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:30:38.491400 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:30:38.491400 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:30:38.491400 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:30:38.491400 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:30:38.492284 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:30:38.492284 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:30:38.492284 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:30:38.492284 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:30:38.492284 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:30:38.492284 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:30:38.492284 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:30:38.492284 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:30:38.492284 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 16 17:30:38.492284 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:30:38.492847 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-8812da5c-0b78-43f7-ae47-4a38808a9634 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:30:38.492847 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:30:38.492847 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:30:38.492847 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:30:38.492847 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:30:38.492847 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:30:38.492847 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:30:38.492847 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:30:38.492847 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:30:38.492847 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:30:38.492847 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:30:38.492847 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:30:38.492847 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:30:38.492847 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:30:38.492847 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:30:38.493595 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:30:38.493595 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:30:38.493595 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:30:38.493595 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:30:38.493595 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:30:38.493595 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:30:38.493595 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:30:38.493595 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:30:38.493595 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:30:38.493595 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:30:38.493595 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:30:38.493595 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:30:38.493595 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:30:38.493595 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:30:38.493595 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:30:38.493595 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:30:38.493595 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:30:38.494468 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:30:38.494468 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:30:38.494468 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:30:38.494468 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:30:38.494690 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 261/501] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jun 16 17:30:38 2022] GET /identity/v3/users/glance => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:30:38.494764 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 241/502] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jun 16 17:30:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:30:38.494839 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967b3eda-430d-41d9-b942-46da90b9aaee None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:38.494906 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8812da5c-0b78-43f7-ae47-4a38808a9634 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:38.516102 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:38.558939 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:38.612113 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967b3eda-430d-41d9-b942-46da90b9aaee None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:38.613333 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8812da5c-0b78-43f7-ae47-4a38808a9634 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:38.653394 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c68332-cea5-44a9-9507-7da872e94f35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:38.654188 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c68332-cea5-44a9-9507-7da872e94f35 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:38.654473 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c68332-cea5-44a9-9507-7da872e94f35 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:38.654752 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c68332-cea5-44a9-9507-7da872e94f35 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:38.657175 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dbc764-c528-4ddf-a7fd-15094ac1076d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:38.658420 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dbc764-c528-4ddf-a7fd-15094ac1076d None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:38.659067 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dbc764-c528-4ddf-a7fd-15094ac1076d None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:38.659351 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dbc764-c528-4ddf-a7fd-15094ac1076d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:38.668712 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c68332-cea5-44a9-9507-7da872e94f35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:38.670310 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c68332-cea5-44a9-9507-7da872e94f35 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:38.672206 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dbc764-c528-4ddf-a7fd-15094ac1076d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:30:38.680652 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-81dbc764-c528-4ddf-a7fd-15094ac1076d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:30:38.680652 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:30:38.680652 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:30:38.680652 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:30:38.680652 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:30:38.680652 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:30:38.680652 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:30:38.680652 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:30:38.680652 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:30:38.680652 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:30:38.680652 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:30:38.680652 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:30:38.680652 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:30:38.680652 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:30:38.680652 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:30:38.680652 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:30:38.681538 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:30:38.681538 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:30:38.681538 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:30:38.681538 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:30:38.681538 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:30:38.681538 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:30:38.681538 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:30:38.681538 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:30:38.681538 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:30:38.681538 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:30:38.681538 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:30:38.681538 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:30:38.681538 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:30:38.681538 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:30:38.681538 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:30:38.681538 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:30:38.682302 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:30:38.682302 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:30:38.682302 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:30:38.682302 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:30:38.682302 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:30:38.682302 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:30:38.682302 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:30:38.682302 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:30:38.682302 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:30:38.682302 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:30:38.682302 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:30:38.682302 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:30:38.682302 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 262/503] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Jun 16 17:30:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:30:38.682944 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dbc764-c528-4ddf-a7fd-15094ac1076d None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:38.692536 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-f1c68332-cea5-44a9-9507-7da872e94f35 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:30:38.692536 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:30:38.692536 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:30:38.692536 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:30:38.692536 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:30:38.692536 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:30:38.692536 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:30:38.692536 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:30:38.692536 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:30:38.692536 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:30:38.692536 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:30:38.692536 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:30:38.692536 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:30:38.692536 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:30:38.692536 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:30:38.693865 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:30:38.693865 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:30:38.693865 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:30:38.693865 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:30:38.693865 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:30:38.693865 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:30:38.693865 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:30:38.693865 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:30:38.693865 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:30:38.693865 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:30:38.693865 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:30:38.693865 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:30:38.693865 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:30:38.693865 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:30:38.693865 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:30:38.693865 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:30:38.693865 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:30:38.694766 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:30:38.694766 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:30:38.694766 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:30:38.694766 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:30:38.694766 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 242/504] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jun 16 17:30:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:30:38.695356 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c68332-cea5-44a9-9507-7da872e94f35 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:38.789704 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:38.809949 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dbc764-c528-4ddf-a7fd-15094ac1076d None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:38.814254 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c68332-cea5-44a9-9507-7da872e94f35 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:38.815965 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:38.851864 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208e3cdc-ec91-4a3c-af86-6985418366fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:38.852840 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208e3cdc-ec91-4a3c-af86-6985418366fe None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:38.853121 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208e3cdc-ec91-4a3c-af86-6985418366fe None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:38.853394 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208e3cdc-ec91-4a3c-af86-6985418366fe None admin] PATH_INFO: `/v3/users/nova` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:38.857136 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f68890-cfb1-4f22-9a9a-9f6d1746578c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:38.857824 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f68890-cfb1-4f22-9a9a-9f6d1746578c None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:38.858115 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f68890-cfb1-4f22-9a9a-9f6d1746578c None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:38.858391 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f68890-cfb1-4f22-9a9a-9f6d1746578c None admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:38.859151 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f68890-cfb1-4f22-9a9a-9f6d1746578c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:38.859466 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f68890-cfb1-4f22-9a9a-9f6d1746578c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:38.861153 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f68890-cfb1-4f22-9a9a-9f6d1746578c None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:38.861446 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208e3cdc-ec91-4a3c-af86-6985418366fe None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:38.862956 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208e3cdc-ec91-4a3c-af86-6985418366fe None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:38.873755 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 243/505] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jun 16 17:30:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:38.874711 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f68890-cfb1-4f22-9a9a-9f6d1746578c None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:38.878698 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-208e3cdc-ec91-4a3c-af86-6985418366fe None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 17:30:38.878698 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:30:38.878698 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:30:38.878698 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:30:38.878698 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:30:38.878698 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:30:38.878698 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:30:38.878698 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:30:38.878698 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:30:38.878698 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:30:38.878698 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:30:38.878698 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:30:38.878698 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:30:38.878698 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:30:38.878698 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:30:38.878698 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:30:38.879784 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:30:38.879784 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:30:38.879784 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:30:38.879784 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:30:38.879784 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:30:38.879784 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:30:38.879784 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:30:38.879784 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:30:38.879784 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:30:38.879784 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:30:38.879784 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:30:38.879784 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:30:38.879784 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:30:38.879784 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:30:38.879784 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:30:38.879784 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:30:38.879784 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:30:38.879784 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:30:38.880846 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:30:38.880846 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:30:38.880846 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:30:38.880846 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:30:38.880846 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:30:38.880846 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:30:38.880846 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:30:38.880846 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:30:38.880846 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 17:30:38.880846 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:30:38.880846 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 263/506] 10.222.0.42 () {56 vars in 1093 bytes} [Thu Jun 16 17:30:38 2022] GET /identity/v3/users/nova => generated 82 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:30:38.881465 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208e3cdc-ec91-4a3c-af86-6985418366fe None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:39.003281 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f68890-cfb1-4f22-9a9a-9f6d1746578c None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:39.013331 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208e3cdc-ec91-4a3c-af86-6985418366fe None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:39.042277 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c46c14-163a-47eb-a033-9ffd7cc61668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:39.042887 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c46c14-163a-47eb-a033-9ffd7cc61668 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:39.043174 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c46c14-163a-47eb-a033-9ffd7cc61668 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:39.043451 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c46c14-163a-47eb-a033-9ffd7cc61668 None admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:39.044288 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c46c14-163a-47eb-a033-9ffd7cc61668 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:39.044645 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c46c14-163a-47eb-a033-9ffd7cc61668 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:39.046193 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c46c14-163a-47eb-a033-9ffd7cc61668 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:39.052659 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b0e623-027a-4ac9-877f-162a331ee037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:39.053419 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b0e623-027a-4ac9-877f-162a331ee037 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:39.053706 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b0e623-027a-4ac9-877f-162a331ee037 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:39.054181 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b0e623-027a-4ac9-877f-162a331ee037 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:39.054996 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b0e623-027a-4ac9-877f-162a331ee037 None admin] RBAC: Adding query filter params (name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:39.055310 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b0e623-027a-4ac9-877f-162a331ee037 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:39.056771 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b0e623-027a-4ac9-877f-162a331ee037 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:39.067556 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 264/507] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Jun 16 17:30:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:39.072937 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b0e623-027a-4ac9-877f-162a331ee037 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:39.075937 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:39.088394 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 244/508] 10.222.0.42 () {56 vars in 1137 bytes} [Thu Jun 16 17:30:38 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:39.089389 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c46c14-163a-47eb-a033-9ffd7cc61668 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:39.115512 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:39.190750 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b0e623-027a-4ac9-877f-162a331ee037 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:39.210681 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c46c14-163a-47eb-a033-9ffd7cc61668 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:39.227637 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda4477b-ccbd-41ff-b551-a8f96d3d42b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:39.228470 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda4477b-ccbd-41ff-b551-a8f96d3d42b0 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:39.228823 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda4477b-ccbd-41ff-b551-a8f96d3d42b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:39.229295 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda4477b-ccbd-41ff-b551-a8f96d3d42b0 None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:39.230211 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda4477b-ccbd-41ff-b551-a8f96d3d42b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:39.230620 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda4477b-ccbd-41ff-b551-a8f96d3d42b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:39.232153 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda4477b-ccbd-41ff-b551-a8f96d3d42b0 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:39.247040 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 265/509] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jun 16 17:30:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:39.247988 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda4477b-ccbd-41ff-b551-a8f96d3d42b0 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:39.252347 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0326f1a9-fbcb-4a71-bc50-e9cdf2d2f986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:39.253209 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0326f1a9-fbcb-4a71-bc50-e9cdf2d2f986 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:39.253532 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0326f1a9-fbcb-4a71-bc50-e9cdf2d2f986 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:39.253803 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0326f1a9-fbcb-4a71-bc50-e9cdf2d2f986 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:39.261733 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0326f1a9-fbcb-4a71-bc50-e9cdf2d2f986 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:39.263235 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0326f1a9-fbcb-4a71-bc50-e9cdf2d2f986 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:39.282326 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-0326f1a9-fbcb-4a71-bc50-e9cdf2d2f986 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 17:30:39.282326 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:30:39.282326 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:30:39.282326 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:30:39.282326 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:30:39.282326 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:30:39.282326 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:30:39.282326 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:30:39.282326 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:30:39.282326 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:30:39.282326 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:30:39.282326 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:30:39.282326 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:30:39.282326 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:30:39.282326 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:30:39.282326 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:30:39.283258 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:30:39.283258 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:30:39.283258 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:30:39.283258 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:30:39.283258 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:30:39.283258 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:30:39.283258 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:30:39.283258 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:30:39.283258 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:30:39.283258 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:30:39.283258 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:30:39.283258 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:30:39.283258 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:30:39.283258 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:30:39.283258 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:30:39.283258 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:30:39.283258 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:30:39.283258 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:30:39.284123 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:30:39.284123 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:30:39.284123 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:30:39.284123 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:30:39.284123 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:30:39.284123 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:30:39.284123 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:30:39.284123 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:30:39.284123 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 17:30:39.284123 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:30:39.284123 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 245/510] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jun 16 17:30:39 2022] GET /identity/v3/users/cinder => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:30:39.284723 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0326f1a9-fbcb-4a71-bc50-e9cdf2d2f986 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:39.322829 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:39.374348 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda4477b-ccbd-41ff-b551-a8f96d3d42b0 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:39.382411 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:39.407624 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0326f1a9-fbcb-4a71-bc50-e9cdf2d2f986 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:39.417372 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7738a265-b90c-4f8d-91f6-ca71d5514415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:39.418052 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7738a265-b90c-4f8d-91f6-ca71d5514415 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:39.418334 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7738a265-b90c-4f8d-91f6-ca71d5514415 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:39.418623 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7738a265-b90c-4f8d-91f6-ca71d5514415 None admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:39.419445 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7738a265-b90c-4f8d-91f6-ca71d5514415 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:39.419762 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7738a265-b90c-4f8d-91f6-ca71d5514415 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:39.421249 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7738a265-b90c-4f8d-91f6-ca71d5514415 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:39.445238 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9a121b-fce6-4946-b479-5fd2ee6a125e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:39.446421 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9a121b-fce6-4946-b479-5fd2ee6a125e None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:39.446962 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9a121b-fce6-4946-b479-5fd2ee6a125e None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:39.447437 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9a121b-fce6-4946-b479-5fd2ee6a125e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:39.459570 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9a121b-fce6-4946-b479-5fd2ee6a125e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:39.461127 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9a121b-fce6-4946-b479-5fd2ee6a125e None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:39.463624 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 266/511] 10.222.0.42 () {56 vars in 1133 bytes} [Thu Jun 16 17:30:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:39.464515 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7738a265-b90c-4f8d-91f6-ca71d5514415 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:39.480777 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-8f9a121b-fce6-4946-b479-5fd2ee6a125e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:30:39.480777 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:30:39.480777 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:30:39.480777 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:30:39.480777 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:30:39.480777 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:30:39.480777 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:30:39.480777 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:30:39.480777 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:30:39.480777 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:30:39.480777 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:30:39.480777 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:30:39.480777 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:30:39.480777 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:30:39.480777 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:30:39.481848 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:30:39.481848 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:30:39.481848 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:30:39.481848 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:30:39.481848 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:30:39.481848 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:30:39.481848 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:30:39.481848 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:30:39.481848 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:30:39.481848 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:30:39.481848 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:30:39.481848 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:30:39.481848 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:30:39.481848 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:30:39.481848 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:30:39.481848 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:30:39.481848 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:30:39.482927 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:30:39.482927 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:30:39.482927 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:30:39.482927 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:30:39.483648 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 246/512] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jun 16 17:30:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:30:39.484640 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9a121b-fce6-4946-b479-5fd2ee6a125e None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:39.584610 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7738a265-b90c-4f8d-91f6-ca71d5514415 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:39.607760 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:39.609612 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9a121b-fce6-4946-b479-5fd2ee6a125e None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:39.629970 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f18da34-3ae7-4e7c-9eaa-2d411714d1d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:39.631059 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f18da34-3ae7-4e7c-9eaa-2d411714d1d3 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:39.631344 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f18da34-3ae7-4e7c-9eaa-2d411714d1d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:39.631627 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f18da34-3ae7-4e7c-9eaa-2d411714d1d3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:39.643902 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:39.651409 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8230d75d-6668-4e80-a9e9-7d5b4625533a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:39.652183 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8230d75d-6668-4e80-a9e9-7d5b4625533a None admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:39.652554 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8230d75d-6668-4e80-a9e9-7d5b4625533a None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:39.652905 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8230d75d-6668-4e80-a9e9-7d5b4625533a None admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:39.653691 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8230d75d-6668-4e80-a9e9-7d5b4625533a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:39.656353 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8230d75d-6668-4e80-a9e9-7d5b4625533a None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:39.680952 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 247/513] 10.222.0.42 () {60 vars in 1132 bytes} [Thu Jun 16 17:30:39 2022] POST /identity/v3/users => generated 247 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:30:39.682256 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8230d75d-6668-4e80-a9e9-7d5b4625533a None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:39.726446 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f18da34-3ae7-4e7c-9eaa-2d411714d1d3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:39.728213 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f18da34-3ae7-4e7c-9eaa-2d411714d1d3 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:39.835904 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8230d75d-6668-4e80-a9e9-7d5b4625533a None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:39.872430 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 267/514] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jun 16 17:30:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:30:39.873251 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f18da34-3ae7-4e7c-9eaa-2d411714d1d3 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:39.886082 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3637e56-1980-4248-a4f1-5504e5b80134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:39.887215 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3637e56-1980-4248-a4f1-5504e5b80134 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:39.887215 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3637e56-1980-4248-a4f1-5504e5b80134 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:39.887388 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3637e56-1980-4248-a4f1-5504e5b80134 None admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:39.888783 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3637e56-1980-4248-a4f1-5504e5b80134 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:39.888783 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3637e56-1980-4248-a4f1-5504e5b80134 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:39.891553 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3637e56-1980-4248-a4f1-5504e5b80134 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:39.911210 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:39.924399 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 248/515] 10.222.0.42 () {56 vars in 1137 bytes} [Thu Jun 16 17:30:39 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:39.924399 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3637e56-1980-4248-a4f1-5504e5b80134 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:39.994036 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:40.025058 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f18da34-3ae7-4e7c-9eaa-2d411714d1d3 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:40.069304 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1e96af-81f4-41da-bb53-eb6565081178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:40.071068 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1e96af-81f4-41da-bb53-eb6565081178 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:40.071358 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1e96af-81f4-41da-bb53-eb6565081178 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:40.071636 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1e96af-81f4-41da-bb53-eb6565081178 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:40.073029 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3637e56-1980-4248-a4f1-5504e5b80134 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:40.122376 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e41eb0-f92c-445a-a3bc-9686b949708c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:40.123027 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e41eb0-f92c-445a-a3bc-9686b949708c None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:40.123306 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e41eb0-f92c-445a-a3bc-9686b949708c None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:40.123624 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e41eb0-f92c-445a-a3bc-9686b949708c None admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:40.124885 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e41eb0-f92c-445a-a3bc-9686b949708c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:40.124885 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e41eb0-f92c-445a-a3bc-9686b949708c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:40.126479 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e41eb0-f92c-445a-a3bc-9686b949708c None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:40.140302 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 249/516] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jun 16 17:30:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:40.145679 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e41eb0-f92c-445a-a3bc-9686b949708c None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:40.164161 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1e96af-81f4-41da-bb53-eb6565081178 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:40.167235 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1e96af-81f4-41da-bb53-eb6565081178 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:40.193259 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:40.273084 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e41eb0-f92c-445a-a3bc-9686b949708c None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:40.286544 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:40.295851 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 268/517] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jun 16 17:30:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:30:40.296791 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1e96af-81f4-41da-bb53-eb6565081178 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:40.321437 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2380f30-049a-49c4-8f93-185fc7cee0eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:40.322372 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2380f30-049a-49c4-8f93-185fc7cee0eb None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:40.322726 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2380f30-049a-49c4-8f93-185fc7cee0eb None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:40.323068 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2380f30-049a-49c4-8f93-185fc7cee0eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:40.338284 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2380f30-049a-49c4-8f93-185fc7cee0eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:40.339778 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2380f30-049a-49c4-8f93-185fc7cee0eb None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:40.365844 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-b2380f30-049a-49c4-8f93-185fc7cee0eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:30:40.365844 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:30:40.365844 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:30:40.365844 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:30:40.365844 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:30:40.365844 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:30:40.365844 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:30:40.365844 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:30:40.365844 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:30:40.365844 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:30:40.365844 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:30:40.365844 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:30:40.365844 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:30:40.365844 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:30:40.365844 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:30:40.366741 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:30:40.366741 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:30:40.366741 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:30:40.366741 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:30:40.366741 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:30:40.366741 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:30:40.366741 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:30:40.366741 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:30:40.366741 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:30:40.366741 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:30:40.366741 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:30:40.366741 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:30:40.366741 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:30:40.366741 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:30:40.366741 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:30:40.366741 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:30:40.366741 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:30:40.367571 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:30:40.367571 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:30:40.367571 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:30:40.367571 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:30:40.369939 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 250/518] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jun 16 17:30:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:30:40.370997 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2380f30-049a-49c4-8f93-185fc7cee0eb None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:40.432571 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1e96af-81f4-41da-bb53-eb6565081178 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:40.464067 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:40.487901 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779bb754-a8b7-41ab-b796-4c4a8d072de3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:40.488682 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779bb754-a8b7-41ab-b796-4c4a8d072de3 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:40.489296 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779bb754-a8b7-41ab-b796-4c4a8d072de3 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:40.489576 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779bb754-a8b7-41ab-b796-4c4a8d072de3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:40.503126 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2380f30-049a-49c4-8f93-185fc7cee0eb None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:40.543451 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904cde4e-713a-4f19-b266-9763fb3254e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:40.544348 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904cde4e-713a-4f19-b266-9763fb3254e4 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:40.544730 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904cde4e-713a-4f19-b266-9763fb3254e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:40.545548 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904cde4e-713a-4f19-b266-9763fb3254e4 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:40.555640 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904cde4e-713a-4f19-b266-9763fb3254e4 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:40.557272 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904cde4e-713a-4f19-b266-9763fb3254e4 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:40.573519 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-904cde4e-713a-4f19-b266-9763fb3254e4 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 16 17:30:40.573519 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:30:40.573519 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:30:40.573519 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:30:40.573519 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:30:40.573519 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:30:40.573519 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:30:40.573519 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:30:40.573519 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:30:40.573519 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:30:40.573519 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:30:40.573519 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:30:40.573519 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:30:40.573519 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:30:40.573519 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:30:40.573519 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:30:40.574488 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:30:40.574488 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:30:40.574488 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:30:40.574488 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:30:40.574488 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:30:40.574488 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:30:40.574488 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:30:40.574488 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:30:40.574488 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:30:40.574488 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:30:40.574488 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:30:40.574488 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:30:40.574488 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:30:40.574488 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:30:40.574488 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:30:40.574488 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:30:40.574488 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:30:40.574488 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:30:40.575356 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:30:40.575356 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:30:40.575356 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:30:40.575356 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:30:40.575356 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:30:40.575356 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:30:40.575356 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:30:40.575356 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:30:40.575356 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 16 17:30:40.575356 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:30:40.575356 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 251/519] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Jun 16 17:30:40 2022] GET /identity/v3/users/neutron => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:30:40.575911 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904cde4e-713a-4f19-b266-9763fb3254e4 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:40.594750 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779bb754-a8b7-41ab-b796-4c4a8d072de3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:40.596448 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779bb754-a8b7-41ab-b796-4c4a8d072de3 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:40.607152 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:40.726024 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904cde4e-713a-4f19-b266-9763fb3254e4 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:40.733801 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 269/520] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jun 16 17:30:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:30:40.734821 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779bb754-a8b7-41ab-b796-4c4a8d072de3 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:40.757085 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:40.773095 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0221e66-9daf-4fb6-bcb7-55c0ff16b746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:40.773903 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0221e66-9daf-4fb6-bcb7-55c0ff16b746 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:40.774188 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0221e66-9daf-4fb6-bcb7-55c0ff16b746 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:40.775172 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0221e66-9daf-4fb6-bcb7-55c0ff16b746 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:40.789181 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0221e66-9daf-4fb6-bcb7-55c0ff16b746 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:40.790994 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0221e66-9daf-4fb6-bcb7-55c0ff16b746 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:40.811988 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-f0221e66-9daf-4fb6-bcb7-55c0ff16b746 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:30:40.811988 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:30:40.811988 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:30:40.811988 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:30:40.811988 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:30:40.811988 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:30:40.811988 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:30:40.811988 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:30:40.811988 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:30:40.811988 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:30:40.811988 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:30:40.811988 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:30:40.811988 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:30:40.811988 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:30:40.811988 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:30:40.813657 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:30:40.813657 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:30:40.813657 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:30:40.813657 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:30:40.813657 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:30:40.813657 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:30:40.813657 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:30:40.813657 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:30:40.813657 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:30:40.813657 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:30:40.813657 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:30:40.813657 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:30:40.813657 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:30:40.813657 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:30:40.813657 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:30:40.813657 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:30:40.813657 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:30:40.814756 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:30:40.814756 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:30:40.814756 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:30:40.814756 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:30:40.814756 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 252/521] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jun 16 17:30:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:30:40.815038 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0221e66-9daf-4fb6-bcb7-55c0ff16b746 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:40.866616 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779bb754-a8b7-41ab-b796-4c4a8d072de3 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:40.911802 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:40.932426 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29239661-7b39-4a37-b613-5bd1e5ecc816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:40.932426 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29239661-7b39-4a37-b613-5bd1e5ecc816 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:40.932426 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29239661-7b39-4a37-b613-5bd1e5ecc816 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:40.932426 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29239661-7b39-4a37-b613-5bd1e5ecc816 None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:40.933146 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29239661-7b39-4a37-b613-5bd1e5ecc816 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:40.933146 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29239661-7b39-4a37-b613-5bd1e5ecc816 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:40.933146 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29239661-7b39-4a37-b613-5bd1e5ecc816 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:40.949288 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 270/522] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jun 16 17:30:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:40.950511 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29239661-7b39-4a37-b613-5bd1e5ecc816 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:40.963296 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0221e66-9daf-4fb6-bcb7-55c0ff16b746 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:41.010356 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b47c0ec-96b1-4141-9f41-4b5034a51f8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:41.011105 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b47c0ec-96b1-4141-9f41-4b5034a51f8e None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:41.011441 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b47c0ec-96b1-4141-9f41-4b5034a51f8e None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:41.011727 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b47c0ec-96b1-4141-9f41-4b5034a51f8e None admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:41.012856 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b47c0ec-96b1-4141-9f41-4b5034a51f8e None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:41.013600 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b47c0ec-96b1-4141-9f41-4b5034a51f8e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:41.016451 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b47c0ec-96b1-4141-9f41-4b5034a51f8e None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:41.044927 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 253/523] 10.222.0.42 () {56 vars in 1139 bytes} [Thu Jun 16 17:30:40 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:41.045944 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b47c0ec-96b1-4141-9f41-4b5034a51f8e None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:41.069198 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:41.104852 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29239661-7b39-4a37-b613-5bd1e5ecc816 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:41.152657 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dfbc5f-2d50-4e59-9fcc-c01c8530e823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:41.153810 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dfbc5f-2d50-4e59-9fcc-c01c8530e823 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:41.154290 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dfbc5f-2d50-4e59-9fcc-c01c8530e823 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:41.154759 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dfbc5f-2d50-4e59-9fcc-c01c8530e823 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:41.169681 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dfbc5f-2d50-4e59-9fcc-c01c8530e823 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:41.171462 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dfbc5f-2d50-4e59-9fcc-c01c8530e823 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:41.195217 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-43dfbc5f-2d50-4e59-9fcc-c01c8530e823 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:30:41.195217 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:30:41.195217 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:30:41.195217 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:30:41.195217 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:30:41.195217 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:30:41.195217 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:30:41.195217 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:30:41.195217 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:30:41.195217 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:30:41.195217 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:30:41.195217 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:30:41.195217 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:30:41.195217 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:30:41.195217 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:30:41.196279 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:30:41.196279 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:30:41.196279 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:30:41.196279 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:30:41.196279 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:30:41.196279 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:30:41.196279 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:30:41.196279 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:30:41.196279 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:30:41.196279 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:30:41.196279 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:30:41.196279 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:30:41.196279 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:30:41.196279 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:30:41.196279 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:30:41.196279 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:30:41.196279 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:30:41.197140 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:30:41.197140 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:30:41.197140 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:30:41.197140 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:30:41.198841 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b47c0ec-96b1-4141-9f41-4b5034a51f8e None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:41.199261 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 271/524] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jun 16 17:30:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:30:41.202062 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dfbc5f-2d50-4e59-9fcc-c01c8530e823 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:41.233865 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:41.244493 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cb0db6-67b6-42b3-b8a6-51f57ded4ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:41.245134 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cb0db6-67b6-42b3-b8a6-51f57ded4ba3 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:41.245422 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cb0db6-67b6-42b3-b8a6-51f57ded4ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:41.245707 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cb0db6-67b6-42b3-b8a6-51f57ded4ba3 None admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:41.246433 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cb0db6-67b6-42b3-b8a6-51f57ded4ba3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:41.246744 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cb0db6-67b6-42b3-b8a6-51f57ded4ba3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:41.248344 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cb0db6-67b6-42b3-b8a6-51f57ded4ba3 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:41.263866 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 254/525] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jun 16 17:30:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:41.265623 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cb0db6-67b6-42b3-b8a6-51f57ded4ba3 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:41.361853 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dfbc5f-2d50-4e59-9fcc-c01c8530e823 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:41.397655 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:41.402037 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cb0db6-67b6-42b3-b8a6-51f57ded4ba3 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:41.405592 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450efebd-5046-4858-88d8-520e2f6a5c7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:41.407301 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450efebd-5046-4858-88d8-520e2f6a5c7c None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:41.407824 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450efebd-5046-4858-88d8-520e2f6a5c7c None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:41.408175 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450efebd-5046-4858-88d8-520e2f6a5c7c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:41.416582 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450efebd-5046-4858-88d8-520e2f6a5c7c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:41.418393 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450efebd-5046-4858-88d8-520e2f6a5c7c None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:41.440459 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-450efebd-5046-4858-88d8-520e2f6a5c7c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:30:41.440459 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:30:41.440459 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:30:41.440459 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:30:41.440459 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:30:41.440459 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:30:41.440459 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:30:41.440459 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:30:41.440459 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:30:41.440459 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:30:41.440459 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:30:41.440459 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:30:41.440459 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:30:41.440459 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:30:41.440459 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:30:41.441352 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:30:41.441352 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:30:41.441352 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:30:41.441352 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:30:41.441352 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:30:41.441352 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:30:41.441352 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:30:41.441352 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:30:41.441352 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:30:41.441352 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:30:41.441352 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:30:41.441352 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:30:41.441352 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:30:41.441352 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:30:41.441352 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:30:41.441352 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:30:41.441352 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:30:41.442181 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:30:41.442181 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:30:41.442181 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:30:41.442181 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:30:41.442181 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:30:41.442181 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:30:41.442181 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:30:41.442181 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:30:41.443489 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 272/526] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jun 16 17:30:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:30:41.444416 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450efebd-5046-4858-88d8-520e2f6a5c7c None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:41.448577 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2a15e2-93d0-4853-b881-2e80dd179134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:41.450549 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2a15e2-93d0-4853-b881-2e80dd179134 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:41.450786 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2a15e2-93d0-4853-b881-2e80dd179134 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:41.451170 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2a15e2-93d0-4853-b881-2e80dd179134 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:41.528847 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:41.544787 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2a15e2-93d0-4853-b881-2e80dd179134 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:41.546350 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2a15e2-93d0-4853-b881-2e80dd179134 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:41.575543 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450efebd-5046-4858-88d8-520e2f6a5c7c None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:41.621873 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3178af69-c637-4587-aa18-7e9dabb2f88f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:41.622777 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3178af69-c637-4587-aa18-7e9dabb2f88f None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:41.623241 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3178af69-c637-4587-aa18-7e9dabb2f88f None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:41.623448 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3178af69-c637-4587-aa18-7e9dabb2f88f None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:41.624177 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3178af69-c637-4587-aa18-7e9dabb2f88f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:41.624555 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3178af69-c637-4587-aa18-7e9dabb2f88f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:41.626561 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3178af69-c637-4587-aa18-7e9dabb2f88f None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:41.646321 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 273/527] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jun 16 17:30:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:41.647505 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3178af69-c637-4587-aa18-7e9dabb2f88f None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:41.684036 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 255/528] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jun 16 17:30:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:30:41.684506 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2a15e2-93d0-4853-b881-2e80dd179134 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:41.728310 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:41.795565 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3178af69-c637-4587-aa18-7e9dabb2f88f None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:41.817292 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:41.835604 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2a15e2-93d0-4853-b881-2e80dd179134 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:41.854638 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c46e47a-92c9-4a7a-9e98-5369373f3365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:41.855698 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c46e47a-92c9-4a7a-9e98-5369373f3365 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:41.856789 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c46e47a-92c9-4a7a-9e98-5369373f3365 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:41.857076 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c46e47a-92c9-4a7a-9e98-5369373f3365 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:41.872357 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c46e47a-92c9-4a7a-9e98-5369373f3365 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:41.874006 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c46e47a-92c9-4a7a-9e98-5369373f3365 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:41.886579 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fb437f-4e22-482e-800f-689aceb17f39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:41.887498 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fb437f-4e22-482e-800f-689aceb17f39 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:41.887879 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fb437f-4e22-482e-800f-689aceb17f39 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:41.888251 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fb437f-4e22-482e-800f-689aceb17f39 None admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:41.890229 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fb437f-4e22-482e-800f-689aceb17f39 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:41.891196 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fb437f-4e22-482e-800f-689aceb17f39 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:41.892814 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fb437f-4e22-482e-800f-689aceb17f39 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:41.896216 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-7c46e47a-92c9-4a7a-9e98-5369373f3365 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:30:41.896216 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:30:41.896216 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:30:41.896216 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:30:41.896216 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:30:41.896216 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:30:41.896216 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:30:41.896216 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:30:41.896216 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:30:41.896216 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:30:41.896216 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:30:41.896216 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:30:41.896216 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:30:41.896216 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:30:41.896216 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:30:41.897310 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:30:41.897310 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:30:41.897310 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:30:41.897310 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:30:41.897310 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:30:41.897310 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:30:41.897310 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:30:41.897310 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:30:41.897310 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:30:41.897310 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:30:41.897310 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:30:41.897310 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:30:41.897310 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:30:41.897310 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:30:41.897310 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:30:41.897310 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:30:41.897310 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:30:41.898374 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:30:41.898374 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:30:41.898374 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:30:41.898374 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:30:41.898374 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:30:41.898374 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:30:41.898374 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:30:41.898374 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:30:41.900604 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 274/529] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jun 16 17:30:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:30:41.901658 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c46e47a-92c9-4a7a-9e98-5369373f3365 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:41.921004 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 256/530] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Jun 16 17:30:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:41.921004 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fb437f-4e22-482e-800f-689aceb17f39 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:42.035381 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c46e47a-92c9-4a7a-9e98-5369373f3365 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:42.051604 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fb437f-4e22-482e-800f-689aceb17f39 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:42.052219 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:42.081807 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813c587e-b976-4ce5-bce0-2dd922047e2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:42.082607 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813c587e-b976-4ce5-bce0-2dd922047e2d None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:42.082887 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813c587e-b976-4ce5-bce0-2dd922047e2d None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:42.083164 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813c587e-b976-4ce5-bce0-2dd922047e2d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:42.091223 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813c587e-b976-4ce5-bce0-2dd922047e2d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:42.092796 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813c587e-b976-4ce5-bce0-2dd922047e2d None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:42.102355 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545360b5-93e0-4c56-9fd2-694def52af67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:42.103231 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545360b5-93e0-4c56-9fd2-694def52af67 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:42.103507 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545360b5-93e0-4c56-9fd2-694def52af67 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:42.103780 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545360b5-93e0-4c56-9fd2-694def52af67 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:42.110380 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-813c587e-b976-4ce5-bce0-2dd922047e2d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:30:42.110380 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:30:42.110380 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:30:42.110380 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:30:42.110380 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:30:42.110380 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:30:42.110380 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:30:42.110380 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:30:42.110380 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:30:42.110380 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:30:42.110380 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:30:42.110380 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:30:42.110380 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:30:42.110380 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:30:42.110380 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:30:42.111249 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:30:42.111249 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:30:42.111249 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:30:42.111249 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:30:42.111249 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:30:42.111249 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:30:42.111249 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:30:42.111249 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:30:42.111249 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:30:42.111249 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:30:42.111249 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:30:42.111249 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:30:42.111249 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:30:42.111249 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:30:42.111249 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:30:42.111249 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:30:42.111249 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:30:42.112113 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:30:42.112113 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:30:42.112113 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:30:42.112113 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:30:42.112113 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:30:42.112113 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:30:42.112113 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:30:42.112113 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:30:42.112113 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 275/531] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jun 16 17:30:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:30:42.112637 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813c587e-b976-4ce5-bce0-2dd922047e2d None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:42.116195 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545360b5-93e0-4c56-9fd2-694def52af67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:42.118560 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545360b5-93e0-4c56-9fd2-694def52af67 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:42.129571 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:42.140002 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-545360b5-93e0-4c56-9fd2-694def52af67 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:30:42.140002 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:30:42.140002 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:30:42.140002 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:30:42.140002 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:30:42.140002 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:30:42.140002 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:30:42.140002 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:30:42.140002 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:30:42.140002 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:30:42.140002 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:30:42.140002 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:30:42.140002 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:30:42.140002 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:30:42.140002 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:30:42.142521 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:30:42.142521 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:30:42.142521 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:30:42.142521 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:30:42.142521 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:30:42.142521 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:30:42.142521 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:30:42.142521 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:30:42.142521 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:30:42.142521 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:30:42.142521 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:30:42.142521 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:30:42.142521 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:30:42.142521 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:30:42.142521 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:30:42.142521 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:30:42.142521 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:30:42.143962 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:30:42.143962 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:30:42.143962 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:30:42.143962 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:30:42.143962 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 257/532] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jun 16 17:30:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:30:42.143962 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545360b5-93e0-4c56-9fd2-694def52af67 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:42.236278 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813c587e-b976-4ce5-bce0-2dd922047e2d None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:42.259387 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545360b5-93e0-4c56-9fd2-694def52af67 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:42.275743 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa135bc5-4a75-4d2d-a7a9-4ab03c9141bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:42.276399 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa135bc5-4a75-4d2d-a7a9-4ab03c9141bf None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:42.276674 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa135bc5-4a75-4d2d-a7a9-4ab03c9141bf None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:42.276949 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa135bc5-4a75-4d2d-a7a9-4ab03c9141bf None admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:42.277912 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa135bc5-4a75-4d2d-a7a9-4ab03c9141bf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:42.278225 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa135bc5-4a75-4d2d-a7a9-4ab03c9141bf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:42.279662 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa135bc5-4a75-4d2d-a7a9-4ab03c9141bf None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:42.296908 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e580e206-4fb1-4845-b65a-5094eec2f340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:42.297658 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e580e206-4fb1-4845-b65a-5094eec2f340 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:42.298014 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e580e206-4fb1-4845-b65a-5094eec2f340 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:42.298363 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e580e206-4fb1-4845-b65a-5094eec2f340 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:42.300471 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 276/533] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Jun 16 17:30:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:42.300626 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e580e206-4fb1-4845-b65a-5094eec2f340 None admin] RBAC: Adding query filter params (role.id=cd7dd29fcdc844c78544402dc30986fb, scope.project.id=8abb87f2f141451a8c6ed832db82be18, user.id=d89029c1b3134df1962c041988bead74) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:42.300626 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e580e206-4fb1-4845-b65a-5094eec2f340 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:42.300771 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b177119-ca88-429e-8ca0-43668008d641 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:42.300771 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b177119-ca88-429e-8ca0-43668008d641 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:42.300903 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b177119-ca88-429e-8ca0-43668008d641 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:42.301536 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e580e206-4fb1-4845-b65a-5094eec2f340 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:42.304475 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 277/534] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:30:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:30:42.304475 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b177119-ca88-429e-8ca0-43668008d641 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:42.320433 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 258/535] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jun 16 17:30:42 2022] GET /identity/v3/role_assignments?user.id=d89029c1b3134df1962c041988bead74&scope.project.id=8abb87f2f141451a8c6ed832db82be18&role.id=cd7dd29fcdc844c78544402dc30986fb => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:42.320433 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e580e206-4fb1-4845-b65a-5094eec2f340 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:42.350117 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:42.407618 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:42.457419 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b177119-ca88-429e-8ca0-43668008d641 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:42.461992 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e580e206-4fb1-4845-b65a-5094eec2f340 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:42.510800 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f167adda-2378-483d-924e-6efcac9225c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:42.511523 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f167adda-2378-483d-924e-6efcac9225c5 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:42.511807 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f167adda-2378-483d-924e-6efcac9225c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:42.512070 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f167adda-2378-483d-924e-6efcac9225c5 None admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:42.512900 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f167adda-2378-483d-924e-6efcac9225c5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:42.513211 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f167adda-2378-483d-924e-6efcac9225c5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:42.513854 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de62fffd-584a-4424-a270-8dc2130eea11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:42.514855 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de62fffd-584a-4424-a270-8dc2130eea11 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:42.514962 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f167adda-2378-483d-924e-6efcac9225c5 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:42.515224 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de62fffd-584a-4424-a270-8dc2130eea11 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:42.516115 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de62fffd-584a-4424-a270-8dc2130eea11 None admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:42.516963 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de62fffd-584a-4424-a270-8dc2130eea11 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:42.517274 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de62fffd-584a-4424-a270-8dc2130eea11 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:42.518669 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de62fffd-584a-4424-a270-8dc2130eea11 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:42.528701 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 278/536] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Jun 16 17:30:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:42.528701 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f167adda-2378-483d-924e-6efcac9225c5 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:42.539970 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 259/537] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jun 16 17:30:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:42.542028 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc28bd5-7bab-46a0-bd30-08dd1d01dd3e None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:42.542028 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc28bd5-7bab-46a0-bd30-08dd1d01dd3e None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:42.542028 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc28bd5-7bab-46a0-bd30-08dd1d01dd3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:42.560076 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-7bc28bd5-7bab-46a0-bd30-08dd1d01dd3e None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:30:42.561473 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-7bc28bd5-7bab-46a0-bd30-08dd1d01dd3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:30:42.660613 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 260/538] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:30:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:30:42.662043 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc28bd5-7bab-46a0-bd30-08dd1d01dd3e None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:42.662173 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:42.677961 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f167adda-2378-483d-924e-6efcac9225c5 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:42.723862 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfab071-8735-4ad8-8640-73f4ea9fce97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:42.725129 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfab071-8735-4ad8-8640-73f4ea9fce97 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:42.725629 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfab071-8735-4ad8-8640-73f4ea9fce97 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:42.726572 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfab071-8735-4ad8-8640-73f4ea9fce97 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:42.727876 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfab071-8735-4ad8-8640-73f4ea9fce97 None admin] RBAC: Adding query filter params (role.id=cd7dd29fcdc844c78544402dc30986fb, scope.project.id=8abb87f2f141451a8c6ed832db82be18, user.id=f86c16179a70474a817be9fda33ece9d) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:42.728412 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfab071-8735-4ad8-8640-73f4ea9fce97 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:42.730760 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfab071-8735-4ad8-8640-73f4ea9fce97 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:42.736648 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:42.742314 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 279/539] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jun 16 17:30:42 2022] GET /identity/v3/role_assignments?user.id=f86c16179a70474a817be9fda33ece9d&scope.project.id=8abb87f2f141451a8c6ed832db82be18&role.id=cd7dd29fcdc844c78544402dc30986fb => generated 259 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:42.744699 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfab071-8735-4ad8-8640-73f4ea9fce97 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:42.820525 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc28bd5-7bab-46a0-bd30-08dd1d01dd3e None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:42.898066 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed5565d-e8cd-4fc1-8d59-d6a0c9317953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:42.898693 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed5565d-e8cd-4fc1-8d59-d6a0c9317953 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:42.898975 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed5565d-e8cd-4fc1-8d59-d6a0c9317953 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:42.899248 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed5565d-e8cd-4fc1-8d59-d6a0c9317953 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:42.900106 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed5565d-e8cd-4fc1-8d59-d6a0c9317953 None admin] RBAC: Adding query filter params (role.id=cd7dd29fcdc844c78544402dc30986fb, scope.project.id=8abb87f2f141451a8c6ed832db82be18, user.id=24d5e2637b7043e08313909d5b06e329) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:42.900471 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed5565d-e8cd-4fc1-8d59-d6a0c9317953 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:42.901992 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed5565d-e8cd-4fc1-8d59-d6a0c9317953 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:42.910659 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 261/540] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jun 16 17:30:42 2022] GET /identity/v3/role_assignments?user.id=24d5e2637b7043e08313909d5b06e329&scope.project.id=8abb87f2f141451a8c6ed832db82be18&role.id=cd7dd29fcdc844c78544402dc30986fb => generated 259 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:42.912028 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed5565d-e8cd-4fc1-8d59-d6a0c9317953 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:42.957782 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfab071-8735-4ad8-8640-73f4ea9fce97 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:43.010424 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2d9d64-5450-409e-9548-16d842169fd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:43.011249 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2d9d64-5450-409e-9548-16d842169fd9 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:43.011546 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2d9d64-5450-409e-9548-16d842169fd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:43.011816 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2d9d64-5450-409e-9548-16d842169fd9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:43.031237 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2d9d64-5450-409e-9548-16d842169fd9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:43.033010 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2d9d64-5450-409e-9548-16d842169fd9 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:43.059091 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-5c2d9d64-5450-409e-9548-16d842169fd9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:30:43.059091 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:30:43.059091 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:30:43.059091 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:30:43.059091 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:30:43.059091 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:30:43.059091 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:30:43.059091 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:30:43.059091 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:30:43.059091 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:30:43.059091 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:30:43.059091 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:30:43.059091 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:30:43.059091 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:30:43.059091 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:30:43.060426 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:30:43.060426 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:30:43.060426 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:30:43.060426 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:30:43.060426 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:30:43.060426 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:30:43.060426 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:30:43.060426 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:30:43.060426 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:30:43.060426 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:30:43.060426 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:30:43.060426 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:30:43.060426 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:30:43.060426 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:30:43.060426 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:30:43.060426 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:30:43.060426 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:30:43.061260 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:30:43.061260 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:30:43.061260 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:30:43.061260 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:30:43.061260 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:30:43.061260 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:30:43.061260 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:30:43.061260 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:30:43.073755 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 280/541] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jun 16 17:30:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:30:43.075132 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2d9d64-5450-409e-9548-16d842169fd9 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:43.095308 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed5565d-e8cd-4fc1-8d59-d6a0c9317953 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:43.121997 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:43.140781 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:43.178847 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a3de12-0b7a-4ae6-bedb-6d8567828786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:43.184397 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a3de12-0b7a-4ae6-bedb-6d8567828786 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:43.184397 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a3de12-0b7a-4ae6-bedb-6d8567828786 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:43.184397 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a3de12-0b7a-4ae6-bedb-6d8567828786 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:43.207186 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a3de12-0b7a-4ae6-bedb-6d8567828786 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:30:43.223101 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-09a3de12-0b7a-4ae6-bedb-6d8567828786 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:30:43.223101 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:30:43.223101 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:30:43.223101 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:30:43.223101 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:30:43.223101 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:30:43.223101 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:30:43.223101 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:30:43.223101 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:30:43.223101 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:30:43.223101 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:30:43.223101 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:30:43.223101 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:30:43.223101 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:30:43.223101 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:30:43.223101 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:30:43.224049 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:30:43.224049 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:30:43.224049 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:30:43.224049 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:30:43.224049 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:30:43.224049 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:30:43.224049 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:30:43.224049 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:30:43.224049 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:30:43.224049 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:30:43.224049 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:30:43.224049 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:30:43.224049 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:30:43.224049 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:30:43.224049 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:30:43.224049 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:30:43.224877 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:30:43.224877 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:30:43.224877 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:30:43.224877 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:30:43.224877 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:30:43.224877 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:30:43.224877 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:30:43.224877 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:30:43.224877 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:30:43.224877 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:30:43.224877 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:30:43.224877 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:30:43.224877 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 262/542] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Jun 16 17:30:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:30:43.234933 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a3de12-0b7a-4ae6-bedb-6d8567828786 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:43.310421 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2d9d64-5450-409e-9548-16d842169fd9 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:43.398241 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bab0fab-c686-426b-91a7-d5745e6669d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:43.398843 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bab0fab-c686-426b-91a7-d5745e6669d9 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:43.399125 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bab0fab-c686-426b-91a7-d5745e6669d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:43.399397 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bab0fab-c686-426b-91a7-d5745e6669d9 None admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:43.400191 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bab0fab-c686-426b-91a7-d5745e6669d9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:43.400538 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bab0fab-c686-426b-91a7-d5745e6669d9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:43.401994 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bab0fab-c686-426b-91a7-d5745e6669d9 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:43.433677 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 281/543] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Jun 16 17:30:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:43.436498 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bab0fab-c686-426b-91a7-d5745e6669d9 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:43.460413 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a3de12-0b7a-4ae6-bedb-6d8567828786 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:43.544769 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:43.554836 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7eceef-38e3-4e4f-ba8a-9f7acea67f5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:43.555590 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7eceef-38e3-4e4f-ba8a-9f7acea67f5a None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:43.555877 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7eceef-38e3-4e4f-ba8a-9f7acea67f5a None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:43.556207 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7eceef-38e3-4e4f-ba8a-9f7acea67f5a None admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:43.558821 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7eceef-38e3-4e4f-ba8a-9f7acea67f5a None admin] RBAC: Adding query filter params (name=service) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:43.559135 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7eceef-38e3-4e4f-ba8a-9f7acea67f5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:43.561654 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7eceef-38e3-4e4f-ba8a-9f7acea67f5a None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:43.592404 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 263/544] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Jun 16 17:30:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:43.592404 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7eceef-38e3-4e4f-ba8a-9f7acea67f5a None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:43.602388 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bab0fab-c686-426b-91a7-d5745e6669d9 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:43.675343 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52793e9a-abdd-42ff-b242-8cd3d75ba565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:43.676035 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52793e9a-abdd-42ff-b242-8cd3d75ba565 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:43.676322 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52793e9a-abdd-42ff-b242-8cd3d75ba565 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:43.676664 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52793e9a-abdd-42ff-b242-8cd3d75ba565 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:43.678587 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52793e9a-abdd-42ff-b242-8cd3d75ba565 None admin] RBAC: Adding query filter params (role.id=cd7dd29fcdc844c78544402dc30986fb, scope.project.id=8abb87f2f141451a8c6ed832db82be18, user.id=f94416103f6245888214a997b014f47a) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:43.679962 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52793e9a-abdd-42ff-b242-8cd3d75ba565 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:43.680585 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52793e9a-abdd-42ff-b242-8cd3d75ba565 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:43.693248 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:43.701849 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 282/545] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jun 16 17:30:43 2022] GET /identity/v3/role_assignments?user.id=f94416103f6245888214a997b014f47a&scope.project.id=8abb87f2f141451a8c6ed832db82be18&role.id=cd7dd29fcdc844c78544402dc30986fb => generated 259 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:43.769315 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7eceef-38e3-4e4f-ba8a-9f7acea67f5a None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:43.811728 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e88f95-dcd7-4fde-947a-c2a34e1b83fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:43.812667 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e88f95-dcd7-4fde-947a-c2a34e1b83fb None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:43.813012 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e88f95-dcd7-4fde-947a-c2a34e1b83fb None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:43.813306 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e88f95-dcd7-4fde-947a-c2a34e1b83fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:43.825752 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e88f95-dcd7-4fde-947a-c2a34e1b83fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:43.827236 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e88f95-dcd7-4fde-947a-c2a34e1b83fb None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:43.847000 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-93e88f95-dcd7-4fde-947a-c2a34e1b83fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:30:43.847000 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:30:43.847000 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:30:43.847000 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:30:43.847000 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:30:43.847000 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:30:43.847000 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:30:43.847000 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:30:43.847000 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:30:43.847000 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:30:43.847000 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:30:43.847000 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:30:43.847000 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:30:43.847000 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:30:43.847000 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:30:43.847922 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:30:43.847922 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:30:43.847922 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:30:43.847922 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:30:43.847922 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:30:43.847922 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:30:43.847922 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:30:43.847922 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:30:43.847922 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:30:43.847922 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:30:43.847922 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:30:43.847922 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:30:43.847922 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:30:43.847922 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:30:43.847922 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:30:43.847922 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:30:43.847922 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:30:43.848776 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:30:43.848776 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:30:43.848776 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:30:43.848776 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:30:43.855592 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 264/546] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jun 16 17:30:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:30:43.859984 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52793e9a-abdd-42ff-b242-8cd3d75ba565 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:43.978806 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52793e9a-abdd-42ff-b242-8cd3d75ba565 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:44.018386 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1c4d0c-5343-42cd-8b77-72950ad13c9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:44.019062 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1c4d0c-5343-42cd-8b77-72950ad13c9d None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:44.019347 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1c4d0c-5343-42cd-8b77-72950ad13c9d None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:44.019628 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1c4d0c-5343-42cd-8b77-72950ad13c9d None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:44.020356 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1c4d0c-5343-42cd-8b77-72950ad13c9d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:44.020691 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1c4d0c-5343-42cd-8b77-72950ad13c9d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:44.022231 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1c4d0c-5343-42cd-8b77-72950ad13c9d None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:44.032294 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:44.043587 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 283/547] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jun 16 17:30:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:44.045197 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e88f95-dcd7-4fde-947a-c2a34e1b83fb None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:44.158407 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:44.171352 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e88f95-dcd7-4fde-947a-c2a34e1b83fb None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:44.222572 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37563f9b-90bc-4094-bf7a-fda992ed124c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:44.223444 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37563f9b-90bc-4094-bf7a-fda992ed124c None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:44.223801 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37563f9b-90bc-4094-bf7a-fda992ed124c None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:44.225268 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37563f9b-90bc-4094-bf7a-fda992ed124c None admin] PATH_INFO: `/v3/users/swift` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:44.239071 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37563f9b-90bc-4094-bf7a-fda992ed124c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:44.243640 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37563f9b-90bc-4094-bf7a-fda992ed124c None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:44.265885 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-37563f9b-90bc-4094-bf7a-fda992ed124c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 16 17:30:44.265885 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:30:44.265885 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:30:44.265885 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:30:44.265885 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:30:44.265885 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:30:44.265885 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:30:44.265885 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:30:44.265885 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:30:44.265885 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:30:44.265885 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:30:44.265885 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:30:44.265885 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:30:44.265885 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:30:44.265885 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:30:44.265885 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:30:44.267892 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:30:44.267892 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:30:44.267892 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:30:44.267892 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:30:44.267892 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:30:44.267892 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:30:44.267892 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:30:44.267892 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:30:44.267892 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:30:44.267892 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:30:44.267892 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:30:44.267892 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:30:44.267892 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:30:44.267892 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:30:44.267892 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:30:44.267892 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:30:44.267892 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:30:44.267892 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:30:44.269071 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:30:44.269071 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:30:44.269071 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:30:44.269071 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:30:44.269071 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:30:44.269071 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:30:44.269071 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:30:44.269071 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:30:44.269071 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 16 17:30:44.269071 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:30:44.276398 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 265/548] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Jun 16 17:30:44 2022] GET /identity/v3/users/swift => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:30:44.285794 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1c4d0c-5343-42cd-8b77-72950ad13c9d None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:44.468402 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1c4d0c-5343-42cd-8b77-72950ad13c9d None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:44.508481 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163b2d1d-d2f8-4f18-96ec-22d24485e005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:44.508664 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163b2d1d-d2f8-4f18-96ec-22d24485e005 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:44.512407 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163b2d1d-d2f8-4f18-96ec-22d24485e005 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:44.512407 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163b2d1d-d2f8-4f18-96ec-22d24485e005 None admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:44.512407 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163b2d1d-d2f8-4f18-96ec-22d24485e005 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:44.512407 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163b2d1d-d2f8-4f18-96ec-22d24485e005 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:44.512407 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163b2d1d-d2f8-4f18-96ec-22d24485e005 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:44.536506 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 284/549] 10.222.0.42 () {56 vars in 1135 bytes} [Thu Jun 16 17:30:44 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:44.541731 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37563f9b-90bc-4094-bf7a-fda992ed124c None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:44.802043 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37563f9b-90bc-4094-bf7a-fda992ed124c None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:44.872397 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef0625c-0025-42fb-acc5-8113cc3d009f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:44.872397 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef0625c-0025-42fb-acc5-8113cc3d009f None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:44.872397 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef0625c-0025-42fb-acc5-8113cc3d009f None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:44.872397 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef0625c-0025-42fb-acc5-8113cc3d009f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:44.926654 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:45.034865 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30864a9-efac-4b93-9442-bbd946a481d8 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:45.035202 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30864a9-efac-4b93-9442-bbd946a481d8 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:45.035490 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30864a9-efac-4b93-9442-bbd946a481d8 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:45.036805 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef0625c-0025-42fb-acc5-8113cc3d009f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:45.038288 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef0625c-0025-42fb-acc5-8113cc3d009f None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:45.040999 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 285/550] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:30:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:30:45.043194 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff318f6-219c-4c95-9c35-3a0f03e51c74 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:45.043516 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff318f6-219c-4c95-9c35-3a0f03e51c74 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:45.043825 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff318f6-219c-4c95-9c35-3a0f03e51c74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:45.070088 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-dff318f6-219c-4c95-9c35-3a0f03e51c74 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:30:45.072058 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-dff318f6-219c-4c95-9c35-3a0f03e51c74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:30:45.142364 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:45.164617 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 286/551] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:30:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 17:30:45.189324 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff318f6-219c-4c95-9c35-3a0f03e51c74 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:45.315813 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff318f6-219c-4c95-9c35-3a0f03e51c74 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:45.363508 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91925535-aff7-4ca0-b0f8-f7b29a9078dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:45.363508 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91925535-aff7-4ca0-b0f8-f7b29a9078dd None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:45.363508 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91925535-aff7-4ca0-b0f8-f7b29a9078dd None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:45.363837 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91925535-aff7-4ca0-b0f8-f7b29a9078dd None admin] PATH_INFO: `/v3/roles/service` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:45.381170 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 266/552] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jun 16 17:30:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 840 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:30:45.393435 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef0625c-0025-42fb-acc5-8113cc3d009f None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:45.393629 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91925535-aff7-4ca0-b0f8-f7b29a9078dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:30:45.401942 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-91925535-aff7-4ca0-b0f8-f7b29a9078dd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:30:45.401942 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:30:45.401942 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:30:45.401942 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:30:45.401942 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:30:45.401942 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:30:45.401942 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:30:45.401942 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:30:45.401942 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:30:45.401942 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:30:45.401942 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:30:45.401942 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:30:45.401942 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:30:45.401942 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:30:45.401942 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:30:45.401942 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:30:45.402998 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:30:45.402998 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:30:45.402998 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:30:45.402998 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:30:45.402998 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:30:45.402998 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:30:45.402998 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:30:45.402998 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:30:45.402998 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:30:45.402998 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:30:45.402998 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:30:45.402998 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:30:45.402998 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:30:45.402998 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:30:45.402998 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:30:45.402998 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:30:45.403827 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:30:45.403827 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:30:45.403827 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:30:45.403827 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:30:45.403827 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:30:45.403827 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:30:45.403827 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:30:45.403827 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:30:45.403827 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:30:45.403827 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:30:45.403827 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:30:45.403827 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:30:45.413593 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 287/553] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Jun 16 17:30:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:30:45.415247 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91925535-aff7-4ca0-b0f8-f7b29a9078dd None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:45.524223 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:45.531512 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:45.637580 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef0625c-0025-42fb-acc5-8113cc3d009f None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:45.641998 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91925535-aff7-4ca0-b0f8-f7b29a9078dd None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:45.725209 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ea28f6-eee8-4a0b-be1f-0a3d649f5c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:45.727013 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ea28f6-eee8-4a0b-be1f-0a3d649f5c8f None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:45.728359 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ea28f6-eee8-4a0b-be1f-0a3d649f5c8f None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:45.729484 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ea28f6-eee8-4a0b-be1f-0a3d649f5c8f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:45.730399 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c4c721-aa7a-4c04-9f25-a9e522573bba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:45.732150 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c4c721-aa7a-4c04-9f25-a9e522573bba None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:45.734914 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c4c721-aa7a-4c04-9f25-a9e522573bba None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:45.735993 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c4c721-aa7a-4c04-9f25-a9e522573bba None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:45.738101 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c4c721-aa7a-4c04-9f25-a9e522573bba None admin] RBAC: Adding query filter params (name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:45.738628 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c4c721-aa7a-4c04-9f25-a9e522573bba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:45.740227 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c4c721-aa7a-4c04-9f25-a9e522573bba None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:45.763012 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 288/554] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Jun 16 17:30:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:30:45.764250 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ea28f6-eee8-4a0b-be1f-0a3d649f5c8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:45.767195 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ea28f6-eee8-4a0b-be1f-0a3d649f5c8f None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:45.771772 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c4c721-aa7a-4c04-9f25-a9e522573bba None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:45.806928 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-91ea28f6-eee8-4a0b-be1f-0a3d649f5c8f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:30:45.806928 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:30:45.806928 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:30:45.806928 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:30:45.806928 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:30:45.806928 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:30:45.806928 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:30:45.806928 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:30:45.806928 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:30:45.806928 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:30:45.806928 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:30:45.806928 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:30:45.806928 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:30:45.806928 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:30:45.806928 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:30:45.807869 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:30:45.807869 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:30:45.807869 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:30:45.807869 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:30:45.807869 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:30:45.807869 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:30:45.807869 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:30:45.807869 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:30:45.807869 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:30:45.807869 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:30:45.807869 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:30:45.807869 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:30:45.807869 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:30:45.807869 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:30:45.807869 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:30:45.807869 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:30:45.807869 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:30:45.808813 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:30:45.808813 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:30:45.808813 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:30:45.808813 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:30:45.816515 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 267/555] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jun 16 17:30:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 419 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:30:45.818357 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b53d9f-d240-4580-a24a-0a554d1ab334 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:45.818857 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b53d9f-d240-4580-a24a-0a554d1ab334 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:45.819343 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b53d9f-d240-4580-a24a-0a554d1ab334 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:45.823732 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 268/556] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:30:45 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:30:45.824915 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b53d9f-d240-4580-a24a-0a554d1ab334 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:45.912507 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c4c721-aa7a-4c04-9f25-a9e522573bba None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:45.926761 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:45.961345 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fecb416-a45e-4c2f-9f95-c1c632644942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:45.962504 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fecb416-a45e-4c2f-9f95-c1c632644942 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:45.963013 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fecb416-a45e-4c2f-9f95-c1c632644942 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:45.963488 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fecb416-a45e-4c2f-9f95-c1c632644942 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:45.975314 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fecb416-a45e-4c2f-9f95-c1c632644942 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:45.977246 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fecb416-a45e-4c2f-9f95-c1c632644942 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:45.985545 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b53d9f-d240-4580-a24a-0a554d1ab334 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:45.988011 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:46.001259 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-4fecb416-a45e-4c2f-9f95-c1c632644942 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:30:46.001259 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:30:46.001259 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:30:46.001259 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:30:46.001259 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:30:46.001259 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:30:46.001259 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:30:46.001259 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:30:46.001259 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:30:46.001259 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:30:46.001259 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:30:46.001259 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:30:46.001259 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:30:46.001259 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:30:46.001259 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:30:46.002229 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:30:46.002229 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:30:46.002229 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:30:46.002229 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:30:46.002229 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:30:46.002229 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:30:46.002229 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:30:46.002229 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:30:46.002229 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:30:46.002229 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:30:46.002229 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:30:46.002229 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:30:46.002229 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:30:46.002229 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:30:46.002229 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:30:46.002229 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:30:46.002229 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:30:46.004000 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:30:46.004000 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:30:46.004000 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:30:46.004000 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:30:46.007363 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 289/557] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jun 16 17:30:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:30:46.009300 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90d79e2-1f9d-42a8-9c75-e6f3a76df18e None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:46.009591 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90d79e2-1f9d-42a8-9c75-e6f3a76df18e None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:46.009868 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90d79e2-1f9d-42a8-9c75-e6f3a76df18e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:46.030433 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-a90d79e2-1f9d-42a8-9c75-e6f3a76df18e None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:30:46.031919 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-a90d79e2-1f9d-42a8-9c75-e6f3a76df18e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:30:46.038414 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa02d880-157b-4139-853a-ce333a7f9ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:46.039225 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa02d880-157b-4139-853a-ce333a7f9ca8 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:46.039713 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa02d880-157b-4139-853a-ce333a7f9ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:46.039991 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa02d880-157b-4139-853a-ce333a7f9ca8 None admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:46.041208 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa02d880-157b-4139-853a-ce333a7f9ca8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:46.041520 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa02d880-157b-4139-853a-ce333a7f9ca8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:46.043274 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa02d880-157b-4139-853a-ce333a7f9ca8 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:46.060158 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 269/558] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jun 16 17:30:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:46.063899 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205569f1-6dc2-4149-9eeb-95030dd95894 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:46.064191 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205569f1-6dc2-4149-9eeb-95030dd95894 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:46.064499 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205569f1-6dc2-4149-9eeb-95030dd95894 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:46.070740 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 270/559] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:30:46 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:30:46.071632 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205569f1-6dc2-4149-9eeb-95030dd95894 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:46.124449 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 290/560] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:30:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:30:46.124449 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90d79e2-1f9d-42a8-9c75-e6f3a76df18e None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:46.210710 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205569f1-6dc2-4149-9eeb-95030dd95894 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:46.259581 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a51cc50-7edc-4efa-ab36-27127179a9c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:46.259983 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:46.260249 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a51cc50-7edc-4efa-ab36-27127179a9c6 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:46.260675 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a51cc50-7edc-4efa-ab36-27127179a9c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:46.260948 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a51cc50-7edc-4efa-ab36-27127179a9c6 None admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:46.261687 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a51cc50-7edc-4efa-ab36-27127179a9c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:46.261988 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a51cc50-7edc-4efa-ab36-27127179a9c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:46.263377 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a51cc50-7edc-4efa-ab36-27127179a9c6 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:46.273776 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90d79e2-1f9d-42a8-9c75-e6f3a76df18e None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:46.282509 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 271/561] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jun 16 17:30:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:46.282509 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd077e17-fc35-413f-8479-86b7eaf1bc2d None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:46.282509 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd077e17-fc35-413f-8479-86b7eaf1bc2d None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:46.282509 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd077e17-fc35-413f-8479-86b7eaf1bc2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:46.303003 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-fd077e17-fc35-413f-8479-86b7eaf1bc2d None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:30:46.304209 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-fd077e17-fc35-413f-8479-86b7eaf1bc2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:30:46.316495 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b630316d-fe14-4cfc-bc8f-e9fcd7903c26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:46.317991 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b630316d-fe14-4cfc-bc8f-e9fcd7903c26 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:46.318483 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b630316d-fe14-4cfc-bc8f-e9fcd7903c26 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:46.318959 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b630316d-fe14-4cfc-bc8f-e9fcd7903c26 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:46.326508 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:46.328549 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b630316d-fe14-4cfc-bc8f-e9fcd7903c26 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:46.330293 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b630316d-fe14-4cfc-bc8f-e9fcd7903c26 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:46.346765 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-b630316d-fe14-4cfc-bc8f-e9fcd7903c26 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:30:46.346765 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:30:46.346765 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:30:46.346765 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:30:46.346765 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:30:46.346765 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:30:46.346765 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:30:46.346765 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:30:46.346765 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:30:46.346765 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:30:46.346765 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:30:46.346765 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:30:46.346765 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:30:46.346765 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:30:46.346765 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:30:46.347858 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:30:46.347858 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:30:46.347858 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:30:46.347858 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:30:46.347858 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:30:46.347858 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:30:46.347858 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:30:46.347858 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:30:46.347858 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:30:46.347858 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:30:46.347858 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:30:46.347858 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:30:46.347858 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:30:46.347858 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:30:46.347858 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:30:46.347858 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:30:46.347858 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:30:46.348763 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:30:46.348763 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:30:46.348763 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:30:46.348763 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:30:46.348763 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:30:46.348763 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:30:46.348763 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:30:46.348763 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:30:46.354086 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 291/562] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jun 16 17:30:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:30:46.355146 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b630316d-fe14-4cfc-bc8f-e9fcd7903c26 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:46.400787 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 272/563] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:30:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:30:46.402460 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd077e17-fc35-413f-8479-86b7eaf1bc2d None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:46.504129 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b630316d-fe14-4cfc-bc8f-e9fcd7903c26 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:46.553711 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200468be-4697-4a43-9fa0-cb205bc24ddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:46.554633 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200468be-4697-4a43-9fa0-cb205bc24ddf None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:46.554913 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200468be-4697-4a43-9fa0-cb205bc24ddf None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:46.555186 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200468be-4697-4a43-9fa0-cb205bc24ddf None admin] PATH_INFO: `/v3/roles/service` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:46.556609 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd077e17-fc35-413f-8479-86b7eaf1bc2d None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:46.567298 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200468be-4697-4a43-9fa0-cb205bc24ddf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:30:46.575456 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-200468be-4697-4a43-9fa0-cb205bc24ddf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:30:46.575456 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:30:46.575456 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:30:46.575456 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:30:46.575456 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:30:46.575456 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:30:46.575456 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:30:46.575456 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:30:46.575456 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:30:46.575456 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:30:46.575456 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:30:46.575456 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:30:46.575456 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:30:46.575456 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:30:46.575456 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:30:46.575456 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:30:46.576447 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:30:46.576447 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:30:46.576447 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:30:46.576447 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:30:46.576447 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:30:46.576447 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:30:46.576447 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:30:46.576447 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:30:46.576447 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:30:46.576447 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:30:46.576447 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:30:46.576447 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:30:46.576447 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:30:46.576447 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:30:46.576447 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:30:46.576447 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:30:46.577906 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:30:46.577906 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:30:46.577906 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:30:46.577906 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:30:46.577906 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:30:46.577906 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:30:46.577906 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:30:46.577906 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:30:46.577906 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:30:46.577906 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:30:46.577906 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:30:46.577906 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:30:46.577906 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 292/564] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Jun 16 17:30:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:30:46.577906 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200468be-4697-4a43-9fa0-cb205bc24ddf None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:46.601019 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:46.613782 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00823f44-801c-4e62-8ef6-10a66285ecb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:46.614597 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00823f44-801c-4e62-8ef6-10a66285ecb8 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:46.614876 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00823f44-801c-4e62-8ef6-10a66285ecb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:46.615147 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00823f44-801c-4e62-8ef6-10a66285ecb8 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:46.625951 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00823f44-801c-4e62-8ef6-10a66285ecb8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:46.627386 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00823f44-801c-4e62-8ef6-10a66285ecb8 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:46.647829 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-00823f44-801c-4e62-8ef6-10a66285ecb8 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 16 17:30:46.647829 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:30:46.647829 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:30:46.647829 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:30:46.647829 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:30:46.647829 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:30:46.647829 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:30:46.647829 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:30:46.647829 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:30:46.647829 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:30:46.647829 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:30:46.647829 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:30:46.647829 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:30:46.647829 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:30:46.647829 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:30:46.647829 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:30:46.648889 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:30:46.648889 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:30:46.648889 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:30:46.648889 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:30:46.648889 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:30:46.648889 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:30:46.648889 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:30:46.648889 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:30:46.648889 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:30:46.648889 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:30:46.648889 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:30:46.648889 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:30:46.648889 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:30:46.648889 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:30:46.648889 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:30:46.648889 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:30:46.648889 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:30:46.648889 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:30:46.650454 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:30:46.650454 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:30:46.650454 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:30:46.650454 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:30:46.650454 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:30:46.650454 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:30:46.650454 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:30:46.650454 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:30:46.650454 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 16 17:30:46.650454 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:30:46.650454 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 273/565] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jun 16 17:30:46 2022] GET /identity/v3/users/glance => generated 84 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:30:46.650454 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00823f44-801c-4e62-8ef6-10a66285ecb8 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:46.672862 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:46.722034 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200468be-4697-4a43-9fa0-cb205bc24ddf None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:46.774030 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bf03e2-e60b-4c12-b4d8-7ebdb4a33f7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:46.774690 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bf03e2-e60b-4c12-b4d8-7ebdb4a33f7e None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:46.775002 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bf03e2-e60b-4c12-b4d8-7ebdb4a33f7e None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:46.775286 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bf03e2-e60b-4c12-b4d8-7ebdb4a33f7e None admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:46.776312 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bf03e2-e60b-4c12-b4d8-7ebdb4a33f7e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:46.776710 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bf03e2-e60b-4c12-b4d8-7ebdb4a33f7e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:46.778382 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bf03e2-e60b-4c12-b4d8-7ebdb4a33f7e None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:46.793748 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 293/566] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Jun 16 17:30:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:46.794743 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00823f44-801c-4e62-8ef6-10a66285ecb8 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:46.797763 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bf03e2-e60b-4c12-b4d8-7ebdb4a33f7e None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:46.835457 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40ef55b-8c00-49cc-bc75-11d041f3189c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:46.836436 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40ef55b-8c00-49cc-bc75-11d041f3189c None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:46.836793 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40ef55b-8c00-49cc-bc75-11d041f3189c None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:46.837141 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40ef55b-8c00-49cc-bc75-11d041f3189c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:46.847612 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40ef55b-8c00-49cc-bc75-11d041f3189c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:30:46.855231 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-b40ef55b-8c00-49cc-bc75-11d041f3189c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:30:46.855231 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:30:46.855231 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:30:46.855231 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:30:46.855231 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:30:46.855231 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:30:46.855231 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:30:46.855231 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:30:46.855231 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:30:46.855231 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:30:46.855231 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:30:46.855231 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:30:46.855231 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:30:46.855231 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:30:46.855231 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:30:46.855231 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:30:46.856163 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:30:46.856163 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:30:46.856163 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:30:46.856163 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:30:46.856163 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:30:46.856163 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:30:46.856163 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:30:46.856163 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:30:46.856163 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:30:46.856163 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:30:46.856163 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:30:46.856163 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:30:46.856163 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:30:46.856163 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:30:46.856163 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:30:46.856163 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:30:46.857410 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:30:46.857410 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:30:46.857410 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:30:46.857410 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:30:46.857410 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:30:46.857410 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:30:46.857410 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:30:46.857410 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:30:46.857410 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:30:46.857410 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:30:46.857410 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:30:46.857410 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:30:46.857410 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 274/567] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Jun 16 17:30:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:30:46.857410 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40ef55b-8c00-49cc-bc75-11d041f3189c None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:46.895434 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:46.933707 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bf03e2-e60b-4c12-b4d8-7ebdb4a33f7e None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:46.945854 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:46.974114 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dd2855-529a-4f77-9691-9be6286f076c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:46.974805 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dd2855-529a-4f77-9691-9be6286f076c None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:46.975017 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40ef55b-8c00-49cc-bc75-11d041f3189c None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:46.975151 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dd2855-529a-4f77-9691-9be6286f076c None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:46.975322 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dd2855-529a-4f77-9691-9be6286f076c None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:46.976060 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dd2855-529a-4f77-9691-9be6286f076c None admin] RBAC: Adding query filter params (name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:46.976363 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dd2855-529a-4f77-9691-9be6286f076c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:46.977752 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dd2855-529a-4f77-9691-9be6286f076c None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:46.987692 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 294/568] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Jun 16 17:30:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:46.989065 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9744d45-e285-4810-b6bf-21ab380bee2a None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:46.989916 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9744d45-e285-4810-b6bf-21ab380bee2a None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:46.989916 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9744d45-e285-4810-b6bf-21ab380bee2a None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:46.991024 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 295/569] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:30:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:30:46.991549 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9744d45-e285-4810-b6bf-21ab380bee2a None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:47.021396 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87918a63-e08a-434f-92fd-f181be0f8ca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:47.022073 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87918a63-e08a-434f-92fd-f181be0f8ca9 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:47.022356 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87918a63-e08a-434f-92fd-f181be0f8ca9 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:47.022656 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87918a63-e08a-434f-92fd-f181be0f8ca9 None admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:47.023575 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87918a63-e08a-434f-92fd-f181be0f8ca9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:47.023892 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87918a63-e08a-434f-92fd-f181be0f8ca9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:47.025409 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87918a63-e08a-434f-92fd-f181be0f8ca9 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:47.043836 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 275/570] 10.222.0.42 () {56 vars in 1137 bytes} [Thu Jun 16 17:30:46 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:47.044800 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87918a63-e08a-434f-92fd-f181be0f8ca9 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:47.125164 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9744d45-e285-4810-b6bf-21ab380bee2a None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:47.164330 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1316b6-a2f3-49a5-8ef5-31bf9f74803e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:47.165135 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1316b6-a2f3-49a5-8ef5-31bf9f74803e None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:47.165418 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1316b6-a2f3-49a5-8ef5-31bf9f74803e None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:47.165693 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1316b6-a2f3-49a5-8ef5-31bf9f74803e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:47.166510 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1316b6-a2f3-49a5-8ef5-31bf9f74803e None admin] RBAC: Adding query filter params (role.id=cd7dd29fcdc844c78544402dc30986fb, scope.project.id=8abb87f2f141451a8c6ed832db82be18, user.id=47b7a9cd615c41d6983a0007b475038c) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:47.166692 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87918a63-e08a-434f-92fd-f181be0f8ca9 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:47.166834 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1316b6-a2f3-49a5-8ef5-31bf9f74803e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:47.168264 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1316b6-a2f3-49a5-8ef5-31bf9f74803e None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:47.171313 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:47.176887 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 296/571] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jun 16 17:30:46 2022] GET /identity/v3/role_assignments?user.id=47b7a9cd615c41d6983a0007b475038c&scope.project.id=8abb87f2f141451a8c6ed832db82be18&role.id=cd7dd29fcdc844c78544402dc30986fb => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:47.177941 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1316b6-a2f3-49a5-8ef5-31bf9f74803e None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:47.212767 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:47.219147 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a96eae-576a-49b5-a01e-967ff309a3a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:47.219835 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a96eae-576a-49b5-a01e-967ff309a3a2 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:47.220156 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a96eae-576a-49b5-a01e-967ff309a3a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:47.220414 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a96eae-576a-49b5-a01e-967ff309a3a2 None admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:47.221180 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a96eae-576a-49b5-a01e-967ff309a3a2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:47.221525 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a96eae-576a-49b5-a01e-967ff309a3a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:47.222891 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a96eae-576a-49b5-a01e-967ff309a3a2 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:47.235118 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 276/572] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Jun 16 17:30:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:47.237713 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416159c5-d696-43c9-8d17-1d6995163741 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:47.239490 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416159c5-d696-43c9-8d17-1d6995163741 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:47.239780 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416159c5-d696-43c9-8d17-1d6995163741 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:47.259752 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-416159c5-d696-43c9-8d17-1d6995163741 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:30:47.261338 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-416159c5-d696-43c9-8d17-1d6995163741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:30:47.333406 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1316b6-a2f3-49a5-8ef5-31bf9f74803e None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:47.362107 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 277/573] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:30:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:30:47.363146 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416159c5-d696-43c9-8d17-1d6995163741 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:47.400039 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c93a11-cd59-4d12-a9b4-02b321b1e02a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:47.401129 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c93a11-cd59-4d12-a9b4-02b321b1e02a None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:47.401514 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c93a11-cd59-4d12-a9b4-02b321b1e02a None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:47.401891 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c93a11-cd59-4d12-a9b4-02b321b1e02a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:47.416305 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c93a11-cd59-4d12-a9b4-02b321b1e02a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:47.418411 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c93a11-cd59-4d12-a9b4-02b321b1e02a None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:47.443573 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-65c93a11-cd59-4d12-a9b4-02b321b1e02a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:30:47.443573 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:30:47.443573 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:30:47.443573 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:30:47.443573 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:30:47.443573 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:30:47.443573 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:30:47.443573 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:30:47.443573 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:30:47.443573 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:30:47.443573 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:30:47.443573 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:30:47.443573 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:30:47.443573 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:30:47.443573 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:30:47.445128 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:30:47.445128 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:30:47.445128 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:30:47.445128 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:30:47.445128 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:30:47.445128 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:30:47.445128 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:30:47.445128 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:30:47.445128 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:30:47.445128 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:30:47.445128 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:30:47.445128 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:30:47.445128 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:30:47.445128 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:30:47.445128 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:30:47.445128 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:30:47.445128 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:30:47.446431 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:30:47.446431 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:30:47.446431 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:30:47.446431 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:30:47.446431 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 297/574] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jun 16 17:30:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:30:47.451752 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c93a11-cd59-4d12-a9b4-02b321b1e02a None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:47.530059 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:47.545551 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416159c5-d696-43c9-8d17-1d6995163741 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:47.581146 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:47.589459 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c93a11-cd59-4d12-a9b4-02b321b1e02a None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:47.596156 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d4719f-1f0e-4c4d-bd40-8637aaa38bb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:47.598131 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d4719f-1f0e-4c4d-bd40-8637aaa38bb6 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:47.598131 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d4719f-1f0e-4c4d-bd40-8637aaa38bb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:47.598131 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d4719f-1f0e-4c4d-bd40-8637aaa38bb6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:47.613685 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d4719f-1f0e-4c4d-bd40-8637aaa38bb6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:47.615609 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d4719f-1f0e-4c4d-bd40-8637aaa38bb6 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:47.643023 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0ebb23-e183-4c96-af81-d9997eba0056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:47.643023 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0ebb23-e183-4c96-af81-d9997eba0056 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:47.643690 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0ebb23-e183-4c96-af81-d9997eba0056 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:47.644190 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0ebb23-e183-4c96-af81-d9997eba0056 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:47.648832 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-e2d4719f-1f0e-4c4d-bd40-8637aaa38bb6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:30:47.648832 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:30:47.648832 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:30:47.648832 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:30:47.648832 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:30:47.648832 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:30:47.648832 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:30:47.648832 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:30:47.648832 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:30:47.648832 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:30:47.648832 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:30:47.648832 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:30:47.648832 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:30:47.648832 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:30:47.648832 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:30:47.649717 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:30:47.649717 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:30:47.649717 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:30:47.649717 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:30:47.649717 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:30:47.649717 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:30:47.649717 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:30:47.649717 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:30:47.649717 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:30:47.649717 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:30:47.649717 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:30:47.649717 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:30:47.649717 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:30:47.649717 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:30:47.649717 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:30:47.649717 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:30:47.649717 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:30:47.650593 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:30:47.650593 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:30:47.650593 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:30:47.650593 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:30:47.650593 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 278/575] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jun 16 17:30:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:30:47.651410 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d4719f-1f0e-4c4d-bd40-8637aaa38bb6 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:47.662030 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0ebb23-e183-4c96-af81-d9997eba0056 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:47.665747 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0ebb23-e183-4c96-af81-d9997eba0056 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:47.685215 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-4a0ebb23-e183-4c96-af81-d9997eba0056 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:30:47.685215 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:30:47.685215 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:30:47.685215 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:30:47.685215 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:30:47.685215 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:30:47.685215 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:30:47.685215 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:30:47.685215 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:30:47.685215 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:30:47.685215 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:30:47.685215 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:30:47.685215 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:30:47.685215 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:30:47.685215 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:30:47.686146 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:30:47.686146 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:30:47.686146 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:30:47.686146 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:30:47.686146 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:30:47.686146 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:30:47.686146 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:30:47.686146 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:30:47.686146 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:30:47.686146 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:30:47.686146 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:30:47.686146 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:30:47.686146 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:30:47.686146 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:30:47.686146 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:30:47.686146 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:30:47.686146 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:30:47.687099 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:30:47.687099 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:30:47.687099 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:30:47.687099 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:30:47.691511 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 298/576] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jun 16 17:30:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:30:47.692294 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0ebb23-e183-4c96-af81-d9997eba0056 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:47.812899 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d4719f-1f0e-4c4d-bd40-8637aaa38bb6 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:47.820194 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0ebb23-e183-4c96-af81-d9997eba0056 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:47.821796 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:47.859496 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39eed05a-d96b-4691-bd25-da887ace202c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:47.863125 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eed05a-d96b-4691-bd25-da887ace202c None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:47.863125 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eed05a-d96b-4691-bd25-da887ace202c None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:47.863125 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eed05a-d96b-4691-bd25-da887ace202c None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:47.863125 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39eed05a-d96b-4691-bd25-da887ace202c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:47.863125 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39eed05a-d96b-4691-bd25-da887ace202c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:47.863701 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39eed05a-d96b-4691-bd25-da887ace202c None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:47.867653 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db16b9c-db3f-4614-adc6-118a71cf79ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:47.868832 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db16b9c-db3f-4614-adc6-118a71cf79ce None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:47.869321 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db16b9c-db3f-4614-adc6-118a71cf79ce None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:47.869799 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db16b9c-db3f-4614-adc6-118a71cf79ce None admin] PATH_INFO: `/v3/roles/service` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:47.877906 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 299/577] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jun 16 17:30:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:47.878779 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39eed05a-d96b-4691-bd25-da887ace202c None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:47.891995 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db16b9c-db3f-4614-adc6-118a71cf79ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:30:47.900083 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-9db16b9c-db3f-4614-adc6-118a71cf79ce None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:30:47.900083 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:30:47.900083 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:30:47.900083 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:30:47.900083 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:30:47.900083 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:30:47.900083 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:30:47.900083 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:30:47.900083 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:30:47.900083 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:30:47.900083 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:30:47.900083 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:30:47.900083 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:30:47.900083 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:30:47.900083 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:30:47.900083 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:30:47.901075 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:30:47.901075 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:30:47.901075 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:30:47.901075 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:30:47.901075 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:30:47.901075 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:30:47.901075 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:30:47.901075 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:30:47.901075 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:30:47.901075 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:30:47.901075 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:30:47.901075 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:30:47.901075 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:30:47.901075 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:30:47.901075 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:30:47.901075 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:30:47.901896 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:30:47.901896 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:30:47.901896 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:30:47.901896 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:30:47.901896 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:30:47.901896 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:30:47.901896 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:30:47.901896 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:30:47.901896 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:30:47.901896 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:30:47.901896 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:30:47.901896 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:30:47.901896 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 279/578] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Jun 16 17:30:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:30:47.902580 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db16b9c-db3f-4614-adc6-118a71cf79ce None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:47.951934 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:48.028290 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39eed05a-d96b-4691-bd25-da887ace202c None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:48.058217 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db16b9c-db3f-4614-adc6-118a71cf79ce None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:48.086823 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84b02ab-da00-4b1d-bc0b-84ed34f1abcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:48.087498 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84b02ab-da00-4b1d-bc0b-84ed34f1abcc None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:48.087771 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84b02ab-da00-4b1d-bc0b-84ed34f1abcc None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:48.088046 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84b02ab-da00-4b1d-bc0b-84ed34f1abcc None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:48.088798 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84b02ab-da00-4b1d-bc0b-84ed34f1abcc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:48.089105 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84b02ab-da00-4b1d-bc0b-84ed34f1abcc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:48.090461 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84b02ab-da00-4b1d-bc0b-84ed34f1abcc None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:48.102376 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 300/579] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jun 16 17:30:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:48.103192 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84b02ab-da00-4b1d-bc0b-84ed34f1abcc None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:48.126869 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46df5ce-6668-49f0-b696-560748d2b808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:48.127783 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46df5ce-6668-49f0-b696-560748d2b808 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:48.128230 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46df5ce-6668-49f0-b696-560748d2b808 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:48.128685 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46df5ce-6668-49f0-b696-560748d2b808 None admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:48.129412 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46df5ce-6668-49f0-b696-560748d2b808 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:48.129715 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46df5ce-6668-49f0-b696-560748d2b808 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:48.131083 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46df5ce-6668-49f0-b696-560748d2b808 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:48.135562 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:48.149808 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 280/580] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jun 16 17:30:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:48.150714 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46df5ce-6668-49f0-b696-560748d2b808 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:48.244444 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84b02ab-da00-4b1d-bc0b-84ed34f1abcc None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:48.291390 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763cf481-0f75-4590-beca-c0a4ae2f612f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:48.292446 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763cf481-0f75-4590-beca-c0a4ae2f612f None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:48.292896 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763cf481-0f75-4590-beca-c0a4ae2f612f None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:48.293361 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763cf481-0f75-4590-beca-c0a4ae2f612f None admin] PATH_INFO: `/v3/users/nova` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:48.301961 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763cf481-0f75-4590-beca-c0a4ae2f612f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:48.303580 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763cf481-0f75-4590-beca-c0a4ae2f612f None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:48.315487 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:48.318457 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-763cf481-0f75-4590-beca-c0a4ae2f612f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 17:30:48.318457 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:30:48.318457 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:30:48.318457 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:30:48.318457 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:30:48.318457 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:30:48.318457 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:30:48.318457 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:30:48.318457 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:30:48.318457 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:30:48.318457 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:30:48.318457 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:30:48.318457 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:30:48.318457 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:30:48.318457 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:30:48.318457 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:30:48.319968 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:30:48.319968 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:30:48.319968 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:30:48.319968 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:30:48.319968 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:30:48.319968 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:30:48.319968 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:30:48.319968 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:30:48.319968 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:30:48.319968 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:30:48.319968 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:30:48.319968 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:30:48.319968 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:30:48.319968 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:30:48.319968 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:30:48.319968 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:30:48.319968 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:30:48.319968 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:30:48.321220 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:30:48.321220 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:30:48.321220 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:30:48.321220 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:30:48.321220 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:30:48.321220 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:30:48.321220 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:30:48.321220 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:30:48.321220 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 17:30:48.321220 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:30:48.321220 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 301/581] 10.222.0.42 () {56 vars in 1093 bytes} [Thu Jun 16 17:30:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:30:48.321220 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763cf481-0f75-4590-beca-c0a4ae2f612f None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:48.335438 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46df5ce-6668-49f0-b696-560748d2b808 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:48.411735 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7eeb95-2271-49f3-8a53-59eceed0457b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:48.413142 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7eeb95-2271-49f3-8a53-59eceed0457b None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:48.413397 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7eeb95-2271-49f3-8a53-59eceed0457b None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:48.413685 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7eeb95-2271-49f3-8a53-59eceed0457b None admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:48.414435 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7eeb95-2271-49f3-8a53-59eceed0457b None admin] RBAC: Adding query filter params (name=service) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:48.414729 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7eeb95-2271-49f3-8a53-59eceed0457b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:48.416111 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7eeb95-2271-49f3-8a53-59eceed0457b None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:48.429273 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 281/582] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Jun 16 17:30:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:48.430117 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7eeb95-2271-49f3-8a53-59eceed0457b None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:48.445156 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:48.473029 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763cf481-0f75-4590-beca-c0a4ae2f612f None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:48.508880 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b14dd8-4e08-4561-935e-8591ce3046be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:48.510001 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b14dd8-4e08-4561-935e-8591ce3046be None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:48.510659 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b14dd8-4e08-4561-935e-8591ce3046be None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:48.511145 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b14dd8-4e08-4561-935e-8591ce3046be None admin] PATH_INFO: `/v3/projects/service` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:48.519392 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b14dd8-4e08-4561-935e-8591ce3046be None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:48.521284 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b14dd8-4e08-4561-935e-8591ce3046be None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:48.536762 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-34b14dd8-4e08-4561-935e-8591ce3046be None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:30:48.536762 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:30:48.536762 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:30:48.536762 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:30:48.536762 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:30:48.536762 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:30:48.536762 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:30:48.536762 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:30:48.536762 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:30:48.536762 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:30:48.536762 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:30:48.536762 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:30:48.536762 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:30:48.536762 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:30:48.536762 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:30:48.537644 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:30:48.537644 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:30:48.537644 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:30:48.537644 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:30:48.537644 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:30:48.537644 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:30:48.537644 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:30:48.537644 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:30:48.537644 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:30:48.537644 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:30:48.537644 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:30:48.537644 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:30:48.537644 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:30:48.537644 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:30:48.537644 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:30:48.537644 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:30:48.537644 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:30:48.538545 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:30:48.538545 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:30:48.538545 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:30:48.538545 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:30:48.538545 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:30:48.538545 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:30:48.538545 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:30:48.538545 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:30:48.545536 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 302/583] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jun 16 17:30:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:30:48.546640 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b14dd8-4e08-4561-935e-8591ce3046be None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:48.636253 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7eeb95-2271-49f3-8a53-59eceed0457b None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:48.666368 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b14dd8-4e08-4561-935e-8591ce3046be None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:48.675592 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105c664b-5baa-484d-af19-e46d3ae4d530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:48.676792 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105c664b-5baa-484d-af19-e46d3ae4d530 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:48.677547 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105c664b-5baa-484d-af19-e46d3ae4d530 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:48.677926 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105c664b-5baa-484d-af19-e46d3ae4d530 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:48.687701 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105c664b-5baa-484d-af19-e46d3ae4d530 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:48.689508 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105c664b-5baa-484d-af19-e46d3ae4d530 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:48.706920 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-105c664b-5baa-484d-af19-e46d3ae4d530 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 17:30:48.706920 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:30:48.706920 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:30:48.706920 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:30:48.706920 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:30:48.706920 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:30:48.706920 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:30:48.706920 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:30:48.706920 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:30:48.706920 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:30:48.706920 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:30:48.706920 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:30:48.706920 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:30:48.706920 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:30:48.706920 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:30:48.706920 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:30:48.708067 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:30:48.708067 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:30:48.708067 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:30:48.708067 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:30:48.708067 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:30:48.708067 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:30:48.708067 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:30:48.708067 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:30:48.708067 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:30:48.708067 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:30:48.708067 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:30:48.708067 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:30:48.708067 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:30:48.708067 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:30:48.708067 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:30:48.708067 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:30:48.708067 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:30:48.708067 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:30:48.709075 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:30:48.709075 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:30:48.709075 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:30:48.709075 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:30:48.709075 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:30:48.709075 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:30:48.709075 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:30:48.709075 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:30:48.709075 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 17:30:48.709075 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:30:48.714473 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dd473c-5f7a-466e-86d0-df6c070a5b08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:48.714582 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 282/584] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jun 16 17:30:48 2022] GET /identity/v3/users/cinder => generated 84 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:30:48.714655 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dd473c-5f7a-466e-86d0-df6c070a5b08 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:48.714655 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dd473c-5f7a-466e-86d0-df6c070a5b08 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:48.714777 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105c664b-5baa-484d-af19-e46d3ae4d530 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:48.714852 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dd473c-5f7a-466e-86d0-df6c070a5b08 None admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:48.714852 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dd473c-5f7a-466e-86d0-df6c070a5b08 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:48.714852 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dd473c-5f7a-466e-86d0-df6c070a5b08 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:48.714852 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dd473c-5f7a-466e-86d0-df6c070a5b08 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:48.733268 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:48.745207 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:48.762549 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 303/585] 10.222.0.42 () {56 vars in 1133 bytes} [Thu Jun 16 17:30:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:48.766225 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dd473c-5f7a-466e-86d0-df6c070a5b08 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:48.878809 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105c664b-5baa-484d-af19-e46d3ae4d530 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:48.914076 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dd473c-5f7a-466e-86d0-df6c070a5b08 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:48.917644 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64978c7-059b-4c07-b68b-0c7249f8f818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:48.918528 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64978c7-059b-4c07-b68b-0c7249f8f818 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:48.918943 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64978c7-059b-4c07-b68b-0c7249f8f818 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:48.919325 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64978c7-059b-4c07-b68b-0c7249f8f818 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:48.932159 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64978c7-059b-4c07-b68b-0c7249f8f818 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:48.933887 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64978c7-059b-4c07-b68b-0c7249f8f818 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:48.953898 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-b64978c7-059b-4c07-b68b-0c7249f8f818 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:30:48.953898 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:30:48.953898 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:30:48.953898 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:30:48.953898 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:30:48.953898 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:30:48.953898 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:30:48.953898 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:30:48.953898 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:30:48.953898 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:30:48.953898 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:30:48.953898 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:30:48.953898 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:30:48.953898 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:30:48.953898 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:30:48.955015 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:30:48.955015 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:30:48.955015 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:30:48.955015 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:30:48.955015 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:30:48.955015 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:30:48.955015 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:30:48.955015 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:30:48.955015 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:30:48.955015 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:30:48.955015 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:30:48.955015 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:30:48.955015 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:30:48.955015 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:30:48.955015 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:30:48.955015 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:30:48.955015 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:30:48.960633 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:30:48.960633 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:30:48.960633 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:30:48.960633 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:30:48.960633 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 283/586] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jun 16 17:30:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:30:48.960633 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64978c7-059b-4c07-b68b-0c7249f8f818 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:48.965458 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c283efd-e6fb-43e8-9fbf-9e9518008383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:48.966211 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c283efd-e6fb-43e8-9fbf-9e9518008383 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:48.966565 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c283efd-e6fb-43e8-9fbf-9e9518008383 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:48.967011 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c283efd-e6fb-43e8-9fbf-9e9518008383 None admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:48.967992 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c283efd-e6fb-43e8-9fbf-9e9518008383 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:48.968322 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c283efd-e6fb-43e8-9fbf-9e9518008383 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:48.970706 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c283efd-e6fb-43e8-9fbf-9e9518008383 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:48.988707 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 304/587] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Jun 16 17:30:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:48.989469 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c283efd-e6fb-43e8-9fbf-9e9518008383 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:49.034703 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:49.056526 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:49.069691 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64978c7-059b-4c07-b68b-0c7249f8f818 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:49.103949 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c283efd-e6fb-43e8-9fbf-9e9518008383 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:49.106727 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6442baef-5502-465e-ae4c-3b0a704bb573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:49.107406 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6442baef-5502-465e-ae4c-3b0a704bb573 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:49.107694 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6442baef-5502-465e-ae4c-3b0a704bb573 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:49.107972 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6442baef-5502-465e-ae4c-3b0a704bb573 None admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:49.108843 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6442baef-5502-465e-ae4c-3b0a704bb573 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:49.109760 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6442baef-5502-465e-ae4c-3b0a704bb573 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:49.111793 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6442baef-5502-465e-ae4c-3b0a704bb573 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:49.138562 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 284/588] 10.222.0.42 () {56 vars in 1137 bytes} [Thu Jun 16 17:30:48 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:49.145565 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6442baef-5502-465e-ae4c-3b0a704bb573 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:49.155708 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f179cd21-2609-40c3-9dbf-78907e42b47b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:49.157409 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f179cd21-2609-40c3-9dbf-78907e42b47b None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:49.157782 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f179cd21-2609-40c3-9dbf-78907e42b47b None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:49.158140 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f179cd21-2609-40c3-9dbf-78907e42b47b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:49.176696 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f179cd21-2609-40c3-9dbf-78907e42b47b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:49.178213 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f179cd21-2609-40c3-9dbf-78907e42b47b None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:49.199558 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-f179cd21-2609-40c3-9dbf-78907e42b47b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:30:49.199558 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:30:49.199558 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:30:49.199558 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:30:49.199558 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:30:49.199558 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:30:49.199558 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:30:49.199558 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:30:49.199558 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:30:49.199558 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:30:49.199558 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:30:49.199558 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:30:49.199558 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:30:49.199558 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:30:49.199558 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:30:49.200917 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:30:49.200917 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:30:49.200917 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:30:49.200917 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:30:49.200917 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:30:49.200917 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:30:49.200917 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:30:49.200917 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:30:49.200917 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:30:49.200917 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:30:49.200917 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:30:49.200917 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:30:49.200917 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:30:49.200917 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:30:49.200917 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:30:49.200917 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:30:49.200917 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:30:49.202149 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:30:49.202149 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:30:49.202149 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:30:49.202149 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:30:49.204805 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 305/589] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jun 16 17:30:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:30:49.205964 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f179cd21-2609-40c3-9dbf-78907e42b47b None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:49.276324 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6442baef-5502-465e-ae4c-3b0a704bb573 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:49.313410 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:49.324841 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed6aeec-fca8-4c23-8da0-2154c4ac4ffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:49.325940 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed6aeec-fca8-4c23-8da0-2154c4ac4ffd None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:49.326342 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed6aeec-fca8-4c23-8da0-2154c4ac4ffd None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:49.326622 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed6aeec-fca8-4c23-8da0-2154c4ac4ffd None admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:49.327347 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed6aeec-fca8-4c23-8da0-2154c4ac4ffd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:49.327647 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed6aeec-fca8-4c23-8da0-2154c4ac4ffd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:49.328078 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f179cd21-2609-40c3-9dbf-78907e42b47b None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:49.329267 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed6aeec-fca8-4c23-8da0-2154c4ac4ffd None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:49.341278 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:49.350199 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 285/590] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jun 16 17:30:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:49.350199 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed6aeec-fca8-4c23-8da0-2154c4ac4ffd None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:49.366070 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8169b6-1bf7-427b-bab7-7ae20592dfb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:49.366944 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8169b6-1bf7-427b-bab7-7ae20592dfb7 None admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:49.367233 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8169b6-1bf7-427b-bab7-7ae20592dfb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:49.367508 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8169b6-1bf7-427b-bab7-7ae20592dfb7 None admin] PATH_INFO: `/v3/projects/8abb87f2f141451a8c6ed832db82be18/users/d89029c1b3134df1962c041988bead74/roles/cd7dd29fcdc844c78544402dc30986fb` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:49.385485 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8169b6-1bf7-427b-bab7-7ae20592dfb7 None admin] RBAC: Authorizing `identity:create_grant(project_id=8abb87f2f141451a8c6ed832db82be18, user_id=d89029c1b3134df1962c041988bead74, role_id=cd7dd29fcdc844c78544402dc30986fb)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:49.386933 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8169b6-1bf7-427b-bab7-7ae20592dfb7 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:49.401094 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 306/591] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Jun 16 17:30:49 2022] PUT /identity/v3/projects/8abb87f2f141451a8c6ed832db82be18/users/d89029c1b3134df1962c041988bead74/roles/cd7dd29fcdc844c78544402dc30986fb => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:30:49.407535 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8169b6-1bf7-427b-bab7-7ae20592dfb7 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:49.484916 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed6aeec-fca8-4c23-8da0-2154c4ac4ffd None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:49.524465 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2921753c-4123-450c-9e80-16eb2995f6c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:49.525288 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2921753c-4123-450c-9e80-16eb2995f6c6 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:49.525570 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2921753c-4123-450c-9e80-16eb2995f6c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:49.525844 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2921753c-4123-450c-9e80-16eb2995f6c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:49.525965 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8169b6-1bf7-427b-bab7-7ae20592dfb7 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:49.546659 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2921753c-4123-450c-9e80-16eb2995f6c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:49.548269 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2921753c-4123-450c-9e80-16eb2995f6c6 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:49.569083 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9769ac0-ba73-4374-ac7d-4b1698ef36ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:49.569766 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9769ac0-ba73-4374-ac7d-4b1698ef36ca None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:49.570034 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9769ac0-ba73-4374-ac7d-4b1698ef36ca None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:49.570351 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9769ac0-ba73-4374-ac7d-4b1698ef36ca None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:49.571037 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9769ac0-ba73-4374-ac7d-4b1698ef36ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:49.571335 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9769ac0-ba73-4374-ac7d-4b1698ef36ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:49.572776 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9769ac0-ba73-4374-ac7d-4b1698ef36ca None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:49.575126 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-2921753c-4123-450c-9e80-16eb2995f6c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:30:49.575126 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:30:49.575126 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:30:49.575126 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:30:49.575126 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:30:49.575126 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:30:49.575126 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:30:49.575126 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:30:49.575126 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:30:49.575126 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:30:49.575126 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:30:49.575126 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:30:49.575126 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:30:49.575126 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:30:49.575126 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:30:49.576086 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:30:49.576086 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:30:49.576086 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:30:49.576086 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:30:49.576086 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:30:49.576086 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:30:49.576086 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:30:49.576086 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:30:49.576086 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:30:49.576086 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:30:49.576086 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:30:49.576086 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:30:49.576086 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:30:49.576086 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:30:49.576086 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:30:49.576086 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:30:49.576086 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:30:49.577101 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:30:49.577101 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:30:49.577101 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:30:49.577101 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:30:49.581994 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 286/592] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jun 16 17:30:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:30:49.582974 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2921753c-4123-450c-9e80-16eb2995f6c6 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:49.592597 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 307/593] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jun 16 17:30:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:49.594348 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3569dc-c30a-4995-9210-fcba65c519e8 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:49.594348 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3569dc-c30a-4995-9210-fcba65c519e8 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:49.594731 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3569dc-c30a-4995-9210-fcba65c519e8 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:49.596108 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 308/594] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:30:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:30:49.596880 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3569dc-c30a-4995-9210-fcba65c519e8 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:49.628051 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:49.640408 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:49.707561 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2921753c-4123-450c-9e80-16eb2995f6c6 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:49.716048 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3569dc-c30a-4995-9210-fcba65c519e8 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:49.743734 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20761de-ab6d-48b7-9e12-90e38d182607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:49.744544 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20761de-ab6d-48b7-9e12-90e38d182607 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:49.744832 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20761de-ab6d-48b7-9e12-90e38d182607 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:49.745109 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20761de-ab6d-48b7-9e12-90e38d182607 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:49.753950 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20761de-ab6d-48b7-9e12-90e38d182607 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:49.754115 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5555a73b-8a37-4a83-8438-0628d5d49c07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:49.754620 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5555a73b-8a37-4a83-8438-0628d5d49c07 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:49.754891 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5555a73b-8a37-4a83-8438-0628d5d49c07 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:49.755162 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5555a73b-8a37-4a83-8438-0628d5d49c07 None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:49.755472 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20761de-ab6d-48b7-9e12-90e38d182607 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:49.755870 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5555a73b-8a37-4a83-8438-0628d5d49c07 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:49.756169 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5555a73b-8a37-4a83-8438-0628d5d49c07 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:49.757555 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5555a73b-8a37-4a83-8438-0628d5d49c07 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:49.775011 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 309/595] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jun 16 17:30:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:49.775235 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-d20761de-ab6d-48b7-9e12-90e38d182607 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 16 17:30:49.775235 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:30:49.775235 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:30:49.775235 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:30:49.775235 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:30:49.775235 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:30:49.775235 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:30:49.775235 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:30:49.775235 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:30:49.775235 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:30:49.775235 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:30:49.775235 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:30:49.775235 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:30:49.775235 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:30:49.775235 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:30:49.775235 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:30:49.776103 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:30:49.776103 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:30:49.776103 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:30:49.776103 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:30:49.776103 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:30:49.776103 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:30:49.776103 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:30:49.776103 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:30:49.776103 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:30:49.776103 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:30:49.776103 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:30:49.776103 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:30:49.776103 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:30:49.776103 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:30:49.776103 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:30:49.776103 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:30:49.776103 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:30:49.776103 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:30:49.777098 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:30:49.777098 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:30:49.777098 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:30:49.777098 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:30:49.777098 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:30:49.777098 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:30:49.777098 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:30:49.777098 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:30:49.777098 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 16 17:30:49.777098 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:30:49.777639 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5555a73b-8a37-4a83-8438-0628d5d49c07 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:49.777710 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 287/596] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Jun 16 17:30:49 2022] GET /identity/v3/users/neutron => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:30:49.777710 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcd5945-90a5-4892-8ac2-0a0f027e00b2 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:49.777710 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcd5945-90a5-4892-8ac2-0a0f027e00b2 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:49.777710 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcd5945-90a5-4892-8ac2-0a0f027e00b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:49.794841 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-dbcd5945-90a5-4892-8ac2-0a0f027e00b2 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:30:49.796143 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-dbcd5945-90a5-4892-8ac2-0a0f027e00b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:30:49.872237 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 288/597] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:30:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:30:49.873145 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcd5945-90a5-4892-8ac2-0a0f027e00b2 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:49.884030 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:49.911020 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:49.918657 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5555a73b-8a37-4a83-8438-0628d5d49c07 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:49.962507 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d266f3b-fbf3-4753-acd3-373675cb4f44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:49.963283 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d266f3b-fbf3-4753-acd3-373675cb4f44 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:49.963707 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d266f3b-fbf3-4753-acd3-373675cb4f44 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:49.964104 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d266f3b-fbf3-4753-acd3-373675cb4f44 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:49.975393 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d266f3b-fbf3-4753-acd3-373675cb4f44 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:49.977626 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d266f3b-fbf3-4753-acd3-373675cb4f44 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:49.994590 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcd5945-90a5-4892-8ac2-0a0f027e00b2 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:49.996020 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-3d266f3b-fbf3-4753-acd3-373675cb4f44 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:30:49.996020 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:30:49.996020 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:30:49.996020 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:30:49.996020 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:30:49.996020 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:30:49.996020 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:30:49.996020 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:30:49.996020 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:30:49.996020 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:30:49.996020 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:30:49.996020 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:30:49.996020 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:30:49.996020 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:30:49.996020 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:30:49.997367 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:30:49.997367 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:30:49.997367 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:30:49.997367 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:30:49.997367 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:30:49.997367 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:30:49.997367 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:30:49.997367 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:30:49.997367 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:30:49.997367 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:30:49.997367 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:30:49.997367 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:30:49.997367 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:30:49.997367 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:30:49.997367 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:30:49.997367 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:30:49.997367 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:30:49.998443 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:30:49.998443 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:30:49.998443 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:30:49.998443 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:30:49.998443 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:30:49.998443 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:30:49.998443 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:30:49.998443 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:30:49.998443 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 310/598] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jun 16 17:30:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:30:49.998443 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d266f3b-fbf3-4753-acd3-373675cb4f44 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:50.050896 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303817eb-871b-4d4a-ac7d-2418b6a28e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:50.051704 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303817eb-871b-4d4a-ac7d-2418b6a28e57 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:50.051988 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303817eb-871b-4d4a-ac7d-2418b6a28e57 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:50.052268 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303817eb-871b-4d4a-ac7d-2418b6a28e57 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:50.060562 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303817eb-871b-4d4a-ac7d-2418b6a28e57 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:50.062019 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303817eb-871b-4d4a-ac7d-2418b6a28e57 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:50.079519 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-303817eb-871b-4d4a-ac7d-2418b6a28e57 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:30:50.079519 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:30:50.079519 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:30:50.079519 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:30:50.079519 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:30:50.079519 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:30:50.079519 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:30:50.079519 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:30:50.079519 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:30:50.079519 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:30:50.079519 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:30:50.079519 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:30:50.079519 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:30:50.079519 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:30:50.079519 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:30:50.080877 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:30:50.080877 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:30:50.080877 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:30:50.080877 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:30:50.080877 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:30:50.080877 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:30:50.080877 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:30:50.080877 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:30:50.080877 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:30:50.080877 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:30:50.080877 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:30:50.080877 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:30:50.080877 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:30:50.080877 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:30:50.080877 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:30:50.080877 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:30:50.080877 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:30:50.081968 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:30:50.081968 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:30:50.081968 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:30:50.081968 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:30:50.081968 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:30:50.081968 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:30:50.081968 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:30:50.081968 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:30:50.085612 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 289/599] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jun 16 17:30:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:30:50.086518 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303817eb-871b-4d4a-ac7d-2418b6a28e57 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:50.135628 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d266f3b-fbf3-4753-acd3-373675cb4f44 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:50.183756 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da7a086-3acf-44db-b97b-0bf2a14d04a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:50.184923 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da7a086-3acf-44db-b97b-0bf2a14d04a1 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:50.185415 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da7a086-3acf-44db-b97b-0bf2a14d04a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:50.185924 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da7a086-3acf-44db-b97b-0bf2a14d04a1 None admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:50.186983 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da7a086-3acf-44db-b97b-0bf2a14d04a1 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:50.187796 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da7a086-3acf-44db-b97b-0bf2a14d04a1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:50.189925 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da7a086-3acf-44db-b97b-0bf2a14d04a1 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:50.201386 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:50.207823 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:50.216791 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303817eb-871b-4d4a-ac7d-2418b6a28e57 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:50.220812 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 311/600] 10.222.0.42 () {56 vars in 1139 bytes} [Thu Jun 16 17:30:49 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:50.221963 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da7a086-3acf-44db-b97b-0bf2a14d04a1 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:50.266488 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4688e07-49ac-4a71-8653-04b3d7a00dd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:50.267288 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4688e07-49ac-4a71-8653-04b3d7a00dd0 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:50.267620 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4688e07-49ac-4a71-8653-04b3d7a00dd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:50.267840 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4688e07-49ac-4a71-8653-04b3d7a00dd0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:50.279208 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4688e07-49ac-4a71-8653-04b3d7a00dd0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:30:50.289917 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-e4688e07-49ac-4a71-8653-04b3d7a00dd0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:30:50.289917 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:30:50.289917 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:30:50.289917 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:30:50.289917 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:30:50.289917 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:30:50.289917 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:30:50.289917 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:30:50.289917 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:30:50.289917 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:30:50.289917 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:30:50.289917 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:30:50.289917 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:30:50.289917 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:30:50.289917 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:30:50.289917 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:30:50.290944 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:30:50.290944 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:30:50.290944 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:30:50.290944 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:30:50.290944 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:30:50.290944 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:30:50.290944 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:30:50.290944 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:30:50.290944 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:30:50.290944 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:30:50.290944 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:30:50.290944 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:30:50.290944 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:30:50.290944 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:30:50.290944 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:30:50.290944 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:30:50.291784 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:30:50.291784 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:30:50.291784 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:30:50.291784 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:30:50.291784 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:30:50.291784 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:30:50.291784 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:30:50.291784 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:30:50.291784 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:30:50.291784 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:30:50.291784 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:30:50.291784 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:30:50.292432 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 290/601] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Jun 16 17:30:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:30:50.293625 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4688e07-49ac-4a71-8653-04b3d7a00dd0 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:50.351542 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da7a086-3acf-44db-b97b-0bf2a14d04a1 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:50.393060 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca2b193-ddd9-48cd-b359-042105b6dee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:50.394196 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca2b193-ddd9-48cd-b359-042105b6dee1 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:50.394658 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca2b193-ddd9-48cd-b359-042105b6dee1 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:50.395121 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca2b193-ddd9-48cd-b359-042105b6dee1 None admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:50.396213 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca2b193-ddd9-48cd-b359-042105b6dee1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:50.396880 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca2b193-ddd9-48cd-b359-042105b6dee1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:50.398448 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca2b193-ddd9-48cd-b359-042105b6dee1 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:50.415739 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 312/602] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Jun 16 17:30:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:50.417287 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca2b193-ddd9-48cd-b359-042105b6dee1 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:50.419309 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4688e07-49ac-4a71-8653-04b3d7a00dd0 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:50.458831 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e933256c-c1b8-4b1a-82b2-8502cdc52329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:50.459497 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e933256c-c1b8-4b1a-82b2-8502cdc52329 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:50.459805 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e933256c-c1b8-4b1a-82b2-8502cdc52329 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:50.460055 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e933256c-c1b8-4b1a-82b2-8502cdc52329 None admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:50.460858 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e933256c-c1b8-4b1a-82b2-8502cdc52329 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:50.461167 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e933256c-c1b8-4b1a-82b2-8502cdc52329 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:50.462593 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e933256c-c1b8-4b1a-82b2-8502cdc52329 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:50.476842 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 291/603] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Jun 16 17:30:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:50.482136 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e933256c-c1b8-4b1a-82b2-8502cdc52329 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:50.488659 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:50.497088 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:50.540349 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca2b193-ddd9-48cd-b359-042105b6dee1 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:50.577768 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48731cf1-741b-4745-8d8b-d1d8185fa7a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:50.579630 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48731cf1-741b-4745-8d8b-d1d8185fa7a1 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:50.579972 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48731cf1-741b-4745-8d8b-d1d8185fa7a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:50.580302 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48731cf1-741b-4745-8d8b-d1d8185fa7a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:50.591691 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e933256c-c1b8-4b1a-82b2-8502cdc52329 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:50.596555 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48731cf1-741b-4745-8d8b-d1d8185fa7a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:50.598012 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48731cf1-741b-4745-8d8b-d1d8185fa7a1 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:50.617757 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-48731cf1-741b-4745-8d8b-d1d8185fa7a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:30:50.617757 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:30:50.617757 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:30:50.617757 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:30:50.617757 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:30:50.617757 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:30:50.617757 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:30:50.617757 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:30:50.617757 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:30:50.617757 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:30:50.617757 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:30:50.617757 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:30:50.617757 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:30:50.617757 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:30:50.617757 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:30:50.618691 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:30:50.618691 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:30:50.618691 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:30:50.618691 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:30:50.618691 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:30:50.618691 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:30:50.618691 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:30:50.618691 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:30:50.618691 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:30:50.618691 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:30:50.618691 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:30:50.618691 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:30:50.618691 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:30:50.618691 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:30:50.618691 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:30:50.618691 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:30:50.618691 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:30:50.619554 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:30:50.619554 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:30:50.619554 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:30:50.619554 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:30:50.619554 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 313/604] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jun 16 17:30:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:30:50.620610 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48731cf1-741b-4745-8d8b-d1d8185fa7a1 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:50.635245 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b13d93-5c65-4a31-ac69-43f552dd5407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:50.635925 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b13d93-5c65-4a31-ac69-43f552dd5407 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:50.636207 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b13d93-5c65-4a31-ac69-43f552dd5407 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:50.636529 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b13d93-5c65-4a31-ac69-43f552dd5407 None admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:50.637276 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b13d93-5c65-4a31-ac69-43f552dd5407 None admin] RBAC: Adding query filter params (name=service) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:50.637582 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b13d93-5c65-4a31-ac69-43f552dd5407 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:50.638940 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b13d93-5c65-4a31-ac69-43f552dd5407 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:50.649042 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 292/605] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Jun 16 17:30:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:50.649879 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b13d93-5c65-4a31-ac69-43f552dd5407 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:50.737346 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48731cf1-741b-4745-8d8b-d1d8185fa7a1 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:50.745642 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:50.753394 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:50.777175 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f383b81-4788-406f-88df-f287c6b1b21c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:50.778064 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f383b81-4788-406f-88df-f287c6b1b21c None admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:50.778222 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b13d93-5c65-4a31-ac69-43f552dd5407 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:50.778366 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f383b81-4788-406f-88df-f287c6b1b21c None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:50.778617 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f383b81-4788-406f-88df-f287c6b1b21c None admin] PATH_INFO: `/v3/projects/8abb87f2f141451a8c6ed832db82be18/users/f86c16179a70474a817be9fda33ece9d/roles/cd7dd29fcdc844c78544402dc30986fb` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:50.795266 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f383b81-4788-406f-88df-f287c6b1b21c None admin] RBAC: Authorizing `identity:create_grant(project_id=8abb87f2f141451a8c6ed832db82be18, user_id=f86c16179a70474a817be9fda33ece9d, role_id=cd7dd29fcdc844c78544402dc30986fb)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:50.797477 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f383b81-4788-406f-88df-f287c6b1b21c None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:50.811019 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 314/606] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Jun 16 17:30:50 2022] PUT /identity/v3/projects/8abb87f2f141451a8c6ed832db82be18/users/f86c16179a70474a817be9fda33ece9d/roles/cd7dd29fcdc844c78544402dc30986fb => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:30:50.811818 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f383b81-4788-406f-88df-f287c6b1b21c None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:50.829884 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dcdf8f-2f0d-435e-a40b-f6555ada54b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:50.830950 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dcdf8f-2f0d-435e-a40b-f6555ada54b4 None admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:50.831308 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dcdf8f-2f0d-435e-a40b-f6555ada54b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:50.831658 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dcdf8f-2f0d-435e-a40b-f6555ada54b4 None admin] PATH_INFO: `/v3/projects/8abb87f2f141451a8c6ed832db82be18/users/24d5e2637b7043e08313909d5b06e329/roles/cd7dd29fcdc844c78544402dc30986fb` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:50.851305 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dcdf8f-2f0d-435e-a40b-f6555ada54b4 None admin] RBAC: Authorizing `identity:create_grant(project_id=8abb87f2f141451a8c6ed832db82be18, user_id=24d5e2637b7043e08313909d5b06e329, role_id=cd7dd29fcdc844c78544402dc30986fb)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:50.853258 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dcdf8f-2f0d-435e-a40b-f6555ada54b4 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:50.874341 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 293/607] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Jun 16 17:30:50 2022] PUT /identity/v3/projects/8abb87f2f141451a8c6ed832db82be18/users/24d5e2637b7043e08313909d5b06e329/roles/cd7dd29fcdc844c78544402dc30986fb => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:30:50.876805 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dcdf8f-2f0d-435e-a40b-f6555ada54b4 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:50.992479 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f383b81-4788-406f-88df-f287c6b1b21c None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:51.046098 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b944a9-a399-42d4-bb87-73157ab84d45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:51.047839 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b944a9-a399-42d4-bb87-73157ab84d45 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:51.048100 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b944a9-a399-42d4-bb87-73157ab84d45 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:51.048311 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b944a9-a399-42d4-bb87-73157ab84d45 None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:51.049127 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b944a9-a399-42d4-bb87-73157ab84d45 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:51.049400 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b944a9-a399-42d4-bb87-73157ab84d45 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:51.050937 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b944a9-a399-42d4-bb87-73157ab84d45 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:51.066205 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 315/608] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jun 16 17:30:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:51.071795 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b944a9-a399-42d4-bb87-73157ab84d45 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:51.105770 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dcdf8f-2f0d-435e-a40b-f6555ada54b4 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:51.108239 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:51.138223 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:51.155371 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46349a7-ef78-4759-b795-711255db6096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:51.157902 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46349a7-ef78-4759-b795-711255db6096 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:51.158198 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46349a7-ef78-4759-b795-711255db6096 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:51.158479 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46349a7-ef78-4759-b795-711255db6096 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:51.173623 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46349a7-ef78-4759-b795-711255db6096 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:51.175938 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46349a7-ef78-4759-b795-711255db6096 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:51.201720 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-d46349a7-ef78-4759-b795-711255db6096 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:30:51.201720 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:30:51.201720 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:30:51.201720 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:30:51.201720 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:30:51.201720 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:30:51.201720 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:30:51.201720 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:30:51.201720 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:30:51.201720 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:30:51.201720 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:30:51.201720 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:30:51.201720 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:30:51.201720 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:30:51.201720 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:30:51.202654 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:30:51.202654 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:30:51.202654 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:30:51.202654 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:30:51.202654 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:30:51.202654 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:30:51.202654 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:30:51.202654 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:30:51.202654 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:30:51.202654 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:30:51.202654 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:30:51.202654 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:30:51.202654 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:30:51.202654 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:30:51.202654 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:30:51.202654 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:30:51.202654 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:30:51.204387 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:30:51.204387 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:30:51.204387 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:30:51.204387 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:30:51.204793 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 294/609] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jun 16 17:30:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:30:51.210262 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46349a7-ef78-4759-b795-711255db6096 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:51.218173 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b944a9-a399-42d4-bb87-73157ab84d45 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:51.284932 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7e86eb-0784-4aac-91a5-3cb136cd7c4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:51.285748 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7e86eb-0784-4aac-91a5-3cb136cd7c4a None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:51.286028 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7e86eb-0784-4aac-91a5-3cb136cd7c4a None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:51.286303 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7e86eb-0784-4aac-91a5-3cb136cd7c4a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:51.298975 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7e86eb-0784-4aac-91a5-3cb136cd7c4a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:51.300339 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7e86eb-0784-4aac-91a5-3cb136cd7c4a None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:51.325112 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-dd7e86eb-0784-4aac-91a5-3cb136cd7c4a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:30:51.325112 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:30:51.325112 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:30:51.325112 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:30:51.325112 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:30:51.325112 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:30:51.325112 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:30:51.325112 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:30:51.325112 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:30:51.325112 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:30:51.325112 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:30:51.325112 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:30:51.325112 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:30:51.325112 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:30:51.325112 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:30:51.326062 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:30:51.326062 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:30:51.326062 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:30:51.326062 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:30:51.326062 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:30:51.326062 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:30:51.326062 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:30:51.326062 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:30:51.326062 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:30:51.326062 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:30:51.326062 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:30:51.326062 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:30:51.326062 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:30:51.326062 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:30:51.326062 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:30:51.326062 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:30:51.326062 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:30:51.326920 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:30:51.326920 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:30:51.326920 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:30:51.326920 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:30:51.326920 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:30:51.326920 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:30:51.326920 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:30:51.326920 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:30:51.329598 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 316/610] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jun 16 17:30:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:30:51.341508 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7e86eb-0784-4aac-91a5-3cb136cd7c4a None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:51.398245 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46349a7-ef78-4759-b795-711255db6096 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:51.455408 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd602ebf-f820-4641-852e-2200a8653c9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:51.458415 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd602ebf-f820-4641-852e-2200a8653c9e None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:51.460069 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd602ebf-f820-4641-852e-2200a8653c9e None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:51.460577 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd602ebf-f820-4641-852e-2200a8653c9e None admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:51.461529 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd602ebf-f820-4641-852e-2200a8653c9e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:51.462273 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd602ebf-f820-4641-852e-2200a8653c9e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:51.464894 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd602ebf-f820-4641-852e-2200a8653c9e None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:51.482055 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 295/611] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jun 16 17:30:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:30:51.487363 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd602ebf-f820-4641-852e-2200a8653c9e None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:51.532251 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:51.555770 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:51.571098 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7e86eb-0784-4aac-91a5-3cb136cd7c4a None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:51.621884 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913b0037-1a71-4eff-9074-4a9a2d260de0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:51.622553 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913b0037-1a71-4eff-9074-4a9a2d260de0 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:51.622836 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913b0037-1a71-4eff-9074-4a9a2d260de0 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:51.623103 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913b0037-1a71-4eff-9074-4a9a2d260de0 None admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:51.623888 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913b0037-1a71-4eff-9074-4a9a2d260de0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:51.624194 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913b0037-1a71-4eff-9074-4a9a2d260de0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:51.626316 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913b0037-1a71-4eff-9074-4a9a2d260de0 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:51.639450 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 317/612] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Jun 16 17:30:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:51.646370 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913b0037-1a71-4eff-9074-4a9a2d260de0 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:51.712396 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd602ebf-f820-4641-852e-2200a8653c9e None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:51.758213 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913b0037-1a71-4eff-9074-4a9a2d260de0 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:51.797586 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d36d180-16a1-4372-9f04-4e5387189a6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:51.798603 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d36d180-16a1-4372-9f04-4e5387189a6a None admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:51.798885 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d36d180-16a1-4372-9f04-4e5387189a6a None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:51.799159 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d36d180-16a1-4372-9f04-4e5387189a6a None admin] PATH_INFO: `/v3/projects/8abb87f2f141451a8c6ed832db82be18/users/f94416103f6245888214a997b014f47a/roles/cd7dd29fcdc844c78544402dc30986fb` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:51.805349 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97193e4-9268-470b-b05a-9c61ac25a93c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:51.806261 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97193e4-9268-470b-b05a-9c61ac25a93c None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:51.806486 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97193e4-9268-470b-b05a-9c61ac25a93c None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:51.806761 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97193e4-9268-470b-b05a-9c61ac25a93c None admin] PATH_INFO: `/v3/users/swift` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:51.814170 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:51.818527 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d36d180-16a1-4372-9f04-4e5387189a6a None admin] RBAC: Authorizing `identity:create_grant(project_id=8abb87f2f141451a8c6ed832db82be18, user_id=f94416103f6245888214a997b014f47a, role_id=cd7dd29fcdc844c78544402dc30986fb)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:51.819992 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d36d180-16a1-4372-9f04-4e5387189a6a None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:51.826067 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97193e4-9268-470b-b05a-9c61ac25a93c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:51.827837 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97193e4-9268-470b-b05a-9c61ac25a93c None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:51.840409 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 318/613] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Jun 16 17:30:51 2022] PUT /identity/v3/projects/8abb87f2f141451a8c6ed832db82be18/users/f94416103f6245888214a997b014f47a/roles/cd7dd29fcdc844c78544402dc30986fb => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:30:51.856559 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-c97193e4-9268-470b-b05a-9c61ac25a93c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 16 17:30:51.856559 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:30:51.856559 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:30:51.856559 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:30:51.856559 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:30:51.856559 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:30:51.856559 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:30:51.856559 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:30:51.856559 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:30:51.856559 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:30:51.856559 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:30:51.856559 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:30:51.856559 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:30:51.856559 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:30:51.856559 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:30:51.856559 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:30:51.857588 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:30:51.857588 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:30:51.857588 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:30:51.857588 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:30:51.857588 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:30:51.857588 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:30:51.857588 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:30:51.857588 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:30:51.857588 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:30:51.857588 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:30:51.857588 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:30:51.857588 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:30:51.857588 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:30:51.857588 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:30:51.857588 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:30:51.857588 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:30:51.857588 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:30:51.857588 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:30:51.858886 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:30:51.858886 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:30:51.858886 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:30:51.858886 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:30:51.858886 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:30:51.858886 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:30:51.858886 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:30:51.858886 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:30:51.858886 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 16 17:30:51.858886 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:30:51.858886 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 296/614] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Jun 16 17:30:51 2022] GET /identity/v3/users/swift => generated 83 bytes in 371 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:30:51.863319 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82ff423-8500-4d13-a99f-85d20c5d92fc None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:51.863719 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82ff423-8500-4d13-a99f-85d20c5d92fc None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:51.863896 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82ff423-8500-4d13-a99f-85d20c5d92fc None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:51.865721 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 319/615] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:30:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:30:51.867865 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97193e4-9268-470b-b05a-9c61ac25a93c None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:51.871376 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d95e0cc-5d97-49a1-a12b-ab2658c184e6 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:51.871839 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d95e0cc-5d97-49a1-a12b-ab2658c184e6 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:51.872571 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d95e0cc-5d97-49a1-a12b-ab2658c184e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:51.909842 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-8d95e0cc-5d97-49a1-a12b-ab2658c184e6 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:30:51.912526 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-8d95e0cc-5d97-49a1-a12b-ab2658c184e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:30:52.003423 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 320/616] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:30:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:30:52.040776 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:52.050889 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97193e4-9268-470b-b05a-9c61ac25a93c None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:52.058480 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d95e0cc-5d97-49a1-a12b-ab2658c184e6 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:52.089854 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d718d99-4219-4a51-b3fa-28f68243b81d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:52.090531 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d718d99-4219-4a51-b3fa-28f68243b81d None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:52.090815 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d718d99-4219-4a51-b3fa-28f68243b81d None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:52.091091 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d718d99-4219-4a51-b3fa-28f68243b81d None admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:52.091962 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d718d99-4219-4a51-b3fa-28f68243b81d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:52.092270 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d718d99-4219-4a51-b3fa-28f68243b81d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:52.094330 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d718d99-4219-4a51-b3fa-28f68243b81d None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:52.115408 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 297/617] 10.222.0.42 () {56 vars in 1135 bytes} [Thu Jun 16 17:30:51 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:52.121693 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d718d99-4219-4a51-b3fa-28f68243b81d None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:52.211937 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d95e0cc-5d97-49a1-a12b-ab2658c184e6 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:52.268762 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:52.293419 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c75a1a7-828b-4a53-a725-3ef62b3de8e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:52.294443 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c75a1a7-828b-4a53-a725-3ef62b3de8e1 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:52.294735 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c75a1a7-828b-4a53-a725-3ef62b3de8e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:52.295016 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c75a1a7-828b-4a53-a725-3ef62b3de8e1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:52.306610 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c75a1a7-828b-4a53-a725-3ef62b3de8e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:30:52.316600 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-4c75a1a7-828b-4a53-a725-3ef62b3de8e1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:30:52.316600 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:30:52.316600 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:30:52.316600 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:30:52.316600 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:30:52.316600 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:30:52.316600 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:30:52.316600 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:30:52.316600 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:30:52.316600 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:30:52.316600 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:30:52.316600 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:30:52.316600 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:30:52.316600 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:30:52.316600 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:30:52.316600 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:30:52.317515 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:30:52.317515 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:30:52.317515 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:30:52.317515 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:30:52.317515 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:30:52.317515 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:30:52.317515 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:30:52.317515 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:30:52.317515 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:30:52.317515 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:30:52.317515 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:30:52.317515 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:30:52.317515 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:30:52.317515 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:30:52.317515 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:30:52.317515 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:30:52.318355 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:30:52.318355 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:30:52.318355 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:30:52.318355 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:30:52.318355 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:30:52.318355 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:30:52.318355 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:30:52.318355 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:30:52.318355 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:30:52.318355 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:30:52.318355 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:30:52.318355 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:30:52.320486 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 321/618] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Jun 16 17:30:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:30:52.323787 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c75a1a7-828b-4a53-a725-3ef62b3de8e1 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:52.325538 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d718d99-4219-4a51-b3fa-28f68243b81d None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:52.438786 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c75a1a7-828b-4a53-a725-3ef62b3de8e1 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:52.441111 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07375af1-7bff-43c6-a707-6491ed9ea1b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:52.441954 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07375af1-7bff-43c6-a707-6491ed9ea1b7 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:52.442242 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07375af1-7bff-43c6-a707-6491ed9ea1b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:52.442515 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07375af1-7bff-43c6-a707-6491ed9ea1b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:52.469535 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07375af1-7bff-43c6-a707-6491ed9ea1b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:52.471056 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07375af1-7bff-43c6-a707-6491ed9ea1b7 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:52.474888 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3722278-b1e1-46f2-a3b7-c0c14d5e9093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:52.475572 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3722278-b1e1-46f2-a3b7-c0c14d5e9093 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:52.475858 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3722278-b1e1-46f2-a3b7-c0c14d5e9093 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:52.476132 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3722278-b1e1-46f2-a3b7-c0c14d5e9093 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:52.476903 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3722278-b1e1-46f2-a3b7-c0c14d5e9093 None admin] RBAC: Adding query filter params (name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:52.477538 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3722278-b1e1-46f2-a3b7-c0c14d5e9093 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:52.478691 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3722278-b1e1-46f2-a3b7-c0c14d5e9093 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:52.491832 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 322/619] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Jun 16 17:30:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:30:52.493107 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3722278-b1e1-46f2-a3b7-c0c14d5e9093 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:52.510543 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-07375af1-7bff-43c6-a707-6491ed9ea1b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:30:52.510543 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:30:52.510543 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:30:52.510543 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:30:52.510543 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:30:52.510543 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:30:52.510543 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:30:52.510543 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:30:52.510543 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:30:52.510543 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:30:52.510543 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:30:52.510543 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:30:52.510543 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:30:52.510543 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:30:52.510543 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:30:52.511491 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:30:52.511491 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:30:52.511491 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:30:52.511491 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:30:52.511491 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:30:52.511491 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:30:52.511491 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:30:52.511491 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:30:52.511491 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:30:52.511491 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:30:52.511491 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:30:52.511491 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:30:52.511491 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:30:52.511491 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:30:52.511491 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:30:52.511491 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:30:52.511491 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:30:52.512362 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:30:52.512362 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:30:52.512362 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:30:52.512362 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:30:52.512362 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 298/620] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jun 16 17:30:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 391 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:30:52.527452 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07375af1-7bff-43c6-a707-6491ed9ea1b7 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:52.537580 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:52.551915 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:52.607786 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3722278-b1e1-46f2-a3b7-c0c14d5e9093 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:52.645378 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e4c189-eceb-4bab-b5ff-fdd25322b3df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:52.646427 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e4c189-eceb-4bab-b5ff-fdd25322b3df None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:52.646890 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e4c189-eceb-4bab-b5ff-fdd25322b3df None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:52.647316 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e4c189-eceb-4bab-b5ff-fdd25322b3df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:52.659037 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e4c189-eceb-4bab-b5ff-fdd25322b3df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:52.660871 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e4c189-eceb-4bab-b5ff-fdd25322b3df None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:52.679991 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-09e4c189-eceb-4bab-b5ff-fdd25322b3df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:30:52.679991 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:30:52.679991 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:30:52.679991 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:30:52.679991 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:30:52.679991 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:30:52.679991 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:30:52.679991 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:30:52.679991 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:30:52.679991 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:30:52.679991 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:30:52.679991 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:30:52.679991 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:30:52.679991 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:30:52.679991 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:30:52.681283 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:30:52.681283 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:30:52.681283 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:30:52.681283 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:30:52.681283 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:30:52.681283 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:30:52.681283 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:30:52.681283 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:30:52.681283 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:30:52.681283 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:30:52.681283 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:30:52.681283 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:30:52.681283 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:30:52.681283 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:30:52.681283 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:30:52.681283 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:30:52.681283 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:30:52.682350 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:30:52.682350 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:30:52.682350 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:30:52.682350 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:30:52.688506 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 323/621] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jun 16 17:30:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:30:52.690539 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e4c189-eceb-4bab-b5ff-fdd25322b3df None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:52.775683 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07375af1-7bff-43c6-a707-6491ed9ea1b7 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:52.823874 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df05d042-c9e0-4de9-a213-2efafa02514f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:52.824577 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df05d042-c9e0-4de9-a213-2efafa02514f None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:52.824864 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df05d042-c9e0-4de9-a213-2efafa02514f None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:52.825368 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df05d042-c9e0-4de9-a213-2efafa02514f None admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:52.825968 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df05d042-c9e0-4de9-a213-2efafa02514f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:52.826372 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df05d042-c9e0-4de9-a213-2efafa02514f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:52.827636 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df05d042-c9e0-4de9-a213-2efafa02514f None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:52.833624 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:52.842782 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 299/622] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jun 16 17:30:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:52.853725 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df05d042-c9e0-4de9-a213-2efafa02514f None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:52.854609 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e4c189-eceb-4bab-b5ff-fdd25322b3df None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:52.950618 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38be0860-9eb1-490f-90c7-de88f5f2c5e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:52.951344 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38be0860-9eb1-490f-90c7-de88f5f2c5e4 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:52.951610 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38be0860-9eb1-490f-90c7-de88f5f2c5e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:52.951899 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38be0860-9eb1-490f-90c7-de88f5f2c5e4 None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:52.952685 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38be0860-9eb1-490f-90c7-de88f5f2c5e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:52.953737 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38be0860-9eb1-490f-90c7-de88f5f2c5e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:52.955148 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38be0860-9eb1-490f-90c7-de88f5f2c5e4 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:52.966177 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:52.990827 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 324/623] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jun 16 17:30:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:30:52.998018 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38be0860-9eb1-490f-90c7-de88f5f2c5e4 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:53.011918 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df05d042-c9e0-4de9-a213-2efafa02514f None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:53.083817 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f608cb86-d8bc-44e6-b04b-ea2372b34e6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:53.084951 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f608cb86-d8bc-44e6-b04b-ea2372b34e6e None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:53.085411 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f608cb86-d8bc-44e6-b04b-ea2372b34e6e None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:53.085883 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f608cb86-d8bc-44e6-b04b-ea2372b34e6e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:53.101731 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f608cb86-d8bc-44e6-b04b-ea2372b34e6e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:53.104599 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f608cb86-d8bc-44e6-b04b-ea2372b34e6e None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:53.125718 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-f608cb86-d8bc-44e6-b04b-ea2372b34e6e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:30:53.125718 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:30:53.125718 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:30:53.125718 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:30:53.125718 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:30:53.125718 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:30:53.125718 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:30:53.125718 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:30:53.125718 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:30:53.125718 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:30:53.125718 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:30:53.125718 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:30:53.125718 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:30:53.125718 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:30:53.125718 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:30:53.127033 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:30:53.127033 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:30:53.127033 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:30:53.127033 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:30:53.127033 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:30:53.127033 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:30:53.127033 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:30:53.127033 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:30:53.127033 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:30:53.127033 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:30:53.127033 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:30:53.127033 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:30:53.127033 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:30:53.127033 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:30:53.127033 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:30:53.127033 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:30:53.127033 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:30:53.128477 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:30:53.128477 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:30:53.128477 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:30:53.128477 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:30:53.128477 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:30:53.128477 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:30:53.128477 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:30:53.128477 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:30:53.131444 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 300/624] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jun 16 17:30:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:30:53.139214 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f608cb86-d8bc-44e6-b04b-ea2372b34e6e None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:53.206711 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38be0860-9eb1-490f-90c7-de88f5f2c5e4 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:53.255334 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f09310-876e-4714-b87f-69f670c262aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:53.256408 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f09310-876e-4714-b87f-69f670c262aa None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:53.258312 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f09310-876e-4714-b87f-69f670c262aa None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:53.258734 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f09310-876e-4714-b87f-69f670c262aa None admin] PATH_INFO: `/v3/users/glance` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:53.273814 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f09310-876e-4714-b87f-69f670c262aa None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:53.275665 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f09310-876e-4714-b87f-69f670c262aa None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:53.307633 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-34f09310-876e-4714-b87f-69f670c262aa None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 16 17:30:53.307633 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:30:53.307633 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:30:53.307633 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:30:53.307633 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:30:53.307633 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:30:53.307633 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:30:53.307633 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:30:53.307633 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:30:53.307633 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:30:53.307633 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:30:53.307633 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:30:53.307633 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:30:53.307633 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:30:53.307633 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:30:53.307633 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:30:53.308857 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:30:53.308857 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:30:53.308857 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:30:53.308857 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:30:53.308857 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:30:53.308857 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:30:53.308857 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:30:53.308857 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:30:53.308857 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:30:53.308857 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:30:53.308857 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:30:53.308857 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:30:53.308857 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:30:53.308857 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:30:53.308857 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:30:53.308857 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:30:53.308857 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:30:53.308857 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:30:53.309990 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:30:53.309990 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:30:53.309990 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:30:53.309990 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:30:53.309990 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:30:53.309990 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:30:53.309990 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:30:53.309990 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:30:53.309990 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 16 17:30:53.309990 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:30:53.316867 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 325/625] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jun 16 17:30:52 2022] GET /identity/v3/users/glance => generated 84 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:30:53.325448 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f09310-876e-4714-b87f-69f670c262aa None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:53.361548 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:53.380301 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f608cb86-d8bc-44e6-b04b-ea2372b34e6e None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:53.408890 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:53.424077 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d62d87-0df3-4180-a24a-b3d7f15a3a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:53.425062 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d62d87-0df3-4180-a24a-b3d7f15a3a0e None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:53.425517 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d62d87-0df3-4180-a24a-b3d7f15a3a0e None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:53.426161 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d62d87-0df3-4180-a24a-b3d7f15a3a0e None admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:53.427122 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d62d87-0df3-4180-a24a-b3d7f15a3a0e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:53.427572 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d62d87-0df3-4180-a24a-b3d7f15a3a0e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:53.429394 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d62d87-0df3-4180-a24a-b3d7f15a3a0e None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:53.444854 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 301/626] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Jun 16 17:30:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:53.457579 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d62d87-0df3-4180-a24a-b3d7f15a3a0e None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:53.555766 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f09310-876e-4714-b87f-69f670c262aa None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:53.595508 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db8e4fa-2651-44d9-bdc8-dc2cf9d08d75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:53.596499 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db8e4fa-2651-44d9-bdc8-dc2cf9d08d75 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:53.596783 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db8e4fa-2651-44d9-bdc8-dc2cf9d08d75 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:53.597051 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db8e4fa-2651-44d9-bdc8-dc2cf9d08d75 None admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:53.597878 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db8e4fa-2651-44d9-bdc8-dc2cf9d08d75 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:53.598185 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db8e4fa-2651-44d9-bdc8-dc2cf9d08d75 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:53.599646 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db8e4fa-2651-44d9-bdc8-dc2cf9d08d75 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:53.609652 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d62d87-0df3-4180-a24a-b3d7f15a3a0e None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:53.618685 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 326/627] 10.222.0.42 () {56 vars in 1137 bytes} [Thu Jun 16 17:30:53 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:53.621305 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380f07bf-f0cc-4366-be68-e5908c71c348 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:53.622012 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380f07bf-f0cc-4366-be68-e5908c71c348 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:53.622518 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380f07bf-f0cc-4366-be68-e5908c71c348 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:53.624758 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 327/628] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:30:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:30:53.626198 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c637199a-865a-4e9e-8dd5-43f71c32e842 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:53.626703 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c637199a-865a-4e9e-8dd5-43f71c32e842 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:53.627198 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c637199a-865a-4e9e-8dd5-43f71c32e842 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:53.632706 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 328/629] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:30:53 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:30:53.633685 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c637199a-865a-4e9e-8dd5-43f71c32e842 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:53.647909 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bd44a3-1edc-441e-b120-d4e2d24d3cf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:53.649021 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bd44a3-1edc-441e-b120-d4e2d24d3cf4 None admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:53.649343 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bd44a3-1edc-441e-b120-d4e2d24d3cf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:53.649615 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bd44a3-1edc-441e-b120-d4e2d24d3cf4 None admin] PATH_INFO: `/v3/projects/8abb87f2f141451a8c6ed832db82be18/users/47b7a9cd615c41d6983a0007b475038c/roles/cd7dd29fcdc844c78544402dc30986fb` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:53.666147 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bd44a3-1edc-441e-b120-d4e2d24d3cf4 None admin] RBAC: Authorizing `identity:create_grant(project_id=8abb87f2f141451a8c6ed832db82be18, user_id=47b7a9cd615c41d6983a0007b475038c, role_id=cd7dd29fcdc844c78544402dc30986fb)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:53.667755 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bd44a3-1edc-441e-b120-d4e2d24d3cf4 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:53.680966 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 302/630] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Jun 16 17:30:53 2022] PUT /identity/v3/projects/8abb87f2f141451a8c6ed832db82be18/users/47b7a9cd615c41d6983a0007b475038c/roles/cd7dd29fcdc844c78544402dc30986fb => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:30:53.682612 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a6c45d-8046-4062-98ea-619b93f21b42 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:53.682971 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a6c45d-8046-4062-98ea-619b93f21b42 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:53.683329 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a6c45d-8046-4062-98ea-619b93f21b42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:53.710288 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-f9a6c45d-8046-4062-98ea-619b93f21b42 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:30:53.711980 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-f9a6c45d-8046-4062-98ea-619b93f21b42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:30:53.735797 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:53.752073 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:53.773038 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c637199a-865a-4e9e-8dd5-43f71c32e842 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:53.812889 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad80f085-a3db-46f7-ba12-087c10200f47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:53.814484 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad80f085-a3db-46f7-ba12-087c10200f47 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:53.815202 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad80f085-a3db-46f7-ba12-087c10200f47 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:53.820572 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 303/631] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:30:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:30:53.820572 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794fe9f4-0f86-48a6-86a9-6f382f79b230 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:53.820572 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794fe9f4-0f86-48a6-86a9-6f382f79b230 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:53.820572 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794fe9f4-0f86-48a6-86a9-6f382f79b230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:53.822076 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad80f085-a3db-46f7-ba12-087c10200f47 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:53.838388 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-794fe9f4-0f86-48a6-86a9-6f382f79b230 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:30:53.844417 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-794fe9f4-0f86-48a6-86a9-6f382f79b230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:30:53.941067 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad80f085-a3db-46f7-ba12-087c10200f47 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:53.942930 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad80f085-a3db-46f7-ba12-087c10200f47 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:53.948316 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 304/632] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:30:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:30:53.949205 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794fe9f4-0f86-48a6-86a9-6f382f79b230 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:54.040583 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:54.069970 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 329/633] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jun 16 17:30:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:30:54.070818 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad80f085-a3db-46f7-ba12-087c10200f47 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:54.087233 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:54.099622 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794fe9f4-0f86-48a6-86a9-6f382f79b230 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:54.152779 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2121f283-463e-4b63-947c-5081dd7c0cc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:54.153879 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2121f283-463e-4b63-947c-5081dd7c0cc2 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:54.154911 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2121f283-463e-4b63-947c-5081dd7c0cc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:54.155340 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2121f283-463e-4b63-947c-5081dd7c0cc2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:54.173467 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2121f283-463e-4b63-947c-5081dd7c0cc2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:30:54.183843 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad80f085-a3db-46f7-ba12-087c10200f47 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:54.184136 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-2121f283-463e-4b63-947c-5081dd7c0cc2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:30:54.184136 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:30:54.184136 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:30:54.184136 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:30:54.184136 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:30:54.184136 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:30:54.184136 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:30:54.184136 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:30:54.184136 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:30:54.184136 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:30:54.184136 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:30:54.184136 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:30:54.184136 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:30:54.184136 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:30:54.184136 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:30:54.184136 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:30:54.185242 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:30:54.185242 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:30:54.185242 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:30:54.185242 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:30:54.185242 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:30:54.185242 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:30:54.185242 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:30:54.185242 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:30:54.185242 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:30:54.185242 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:30:54.185242 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:30:54.185242 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:30:54.185242 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:30:54.185242 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:30:54.185242 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:30:54.185242 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:30:54.186272 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:30:54.186272 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:30:54.186272 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:30:54.186272 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:30:54.186272 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:30:54.186272 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:30:54.186272 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:30:54.186272 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:30:54.186272 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:30:54.186272 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:30:54.186272 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:30:54.186272 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:30:54.194566 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 305/634] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Jun 16 17:30:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:30:54.194566 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2121f283-463e-4b63-947c-5081dd7c0cc2 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:54.236899 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053c0cf0-1325-41c1-8962-b9615a0e0c03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:54.238098 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053c0cf0-1325-41c1-8962-b9615a0e0c03 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:54.238581 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053c0cf0-1325-41c1-8962-b9615a0e0c03 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:54.239050 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053c0cf0-1325-41c1-8962-b9615a0e0c03 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:54.254167 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053c0cf0-1325-41c1-8962-b9615a0e0c03 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:30:54.270336 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-053c0cf0-1325-41c1-8962-b9615a0e0c03 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:30:54.270336 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:30:54.270336 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:30:54.270336 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:30:54.270336 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:30:54.270336 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:30:54.270336 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:30:54.270336 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:30:54.270336 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:30:54.270336 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:30:54.270336 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:30:54.270336 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:30:54.270336 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:30:54.270336 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:30:54.270336 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:30:54.270336 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:30:54.271317 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:30:54.271317 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:30:54.271317 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:30:54.271317 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:30:54.271317 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:30:54.271317 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:30:54.271317 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:30:54.271317 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:30:54.271317 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:30:54.271317 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:30:54.271317 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:30:54.271317 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:30:54.271317 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:30:54.271317 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:30:54.271317 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:30:54.271317 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:30:54.272155 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:30:54.272155 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:30:54.272155 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:30:54.272155 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:30:54.272155 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:30:54.272155 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:30:54.272155 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:30:54.272155 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:30:54.272155 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:30:54.272155 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:30:54.272155 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:30:54.272155 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:30:54.272155 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 330/635] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Jun 16 17:30:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:30:54.272881 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053c0cf0-1325-41c1-8962-b9615a0e0c03 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:54.314447 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:54.337660 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2121f283-463e-4b63-947c-5081dd7c0cc2 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:54.382736 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdeb43a-6fd0-438e-942a-3eaed246a0f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:54.383927 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdeb43a-6fd0-438e-942a-3eaed246a0f8 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:54.384539 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdeb43a-6fd0-438e-942a-3eaed246a0f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:54.385034 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdeb43a-6fd0-438e-942a-3eaed246a0f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:54.391941 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053c0cf0-1325-41c1-8962-b9615a0e0c03 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:54.398070 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdeb43a-6fd0-438e-942a-3eaed246a0f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:54.399541 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdeb43a-6fd0-438e-942a-3eaed246a0f8 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:54.417649 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-5fdeb43a-6fd0-438e-942a-3eaed246a0f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:30:54.417649 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:30:54.417649 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:30:54.417649 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:30:54.417649 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:30:54.417649 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:30:54.417649 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:30:54.417649 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:30:54.417649 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:30:54.417649 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:30:54.417649 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:30:54.417649 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:30:54.417649 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:30:54.417649 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:30:54.417649 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:30:54.418613 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:30:54.418613 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:30:54.418613 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:30:54.418613 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:30:54.418613 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:30:54.418613 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:30:54.418613 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:30:54.418613 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:30:54.418613 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:30:54.418613 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:30:54.418613 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:30:54.418613 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:30:54.418613 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:30:54.418613 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:30:54.418613 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:30:54.418613 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:30:54.418613 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:30:54.419525 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:30:54.419525 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:30:54.419525 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:30:54.419525 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:30:54.419525 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 306/636] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jun 16 17:30:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:30:54.419797 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdeb43a-6fd0-438e-942a-3eaed246a0f8 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:54.430158 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:54.435117 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535be626-7226-4c35-9508-378ee004d098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:54.435802 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535be626-7226-4c35-9508-378ee004d098 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:54.436104 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535be626-7226-4c35-9508-378ee004d098 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:54.436393 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535be626-7226-4c35-9508-378ee004d098 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:54.437198 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535be626-7226-4c35-9508-378ee004d098 None admin] RBAC: Adding query filter params (name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:54.437503 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535be626-7226-4c35-9508-378ee004d098 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:54.439013 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535be626-7226-4c35-9508-378ee004d098 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:54.447649 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 331/637] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Jun 16 17:30:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:54.448746 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3940d52b-16bd-4a7f-b8a3-48cc9498f212 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:54.449035 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3940d52b-16bd-4a7f-b8a3-48cc9498f212 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:54.449310 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3940d52b-16bd-4a7f-b8a3-48cc9498f212 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:54.450689 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 332/638] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:30:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:30:54.454009 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3940d52b-16bd-4a7f-b8a3-48cc9498f212 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:54.534396 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdeb43a-6fd0-438e-942a-3eaed246a0f8 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:54.559535 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:54.567873 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3940d52b-16bd-4a7f-b8a3-48cc9498f212 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:54.571516 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35921272-e926-4725-b9fa-ec9b885fab37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:54.572185 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35921272-e926-4725-b9fa-ec9b885fab37 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:54.572494 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35921272-e926-4725-b9fa-ec9b885fab37 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:54.572770 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35921272-e926-4725-b9fa-ec9b885fab37 None admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:54.573506 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35921272-e926-4725-b9fa-ec9b885fab37 None admin] RBAC: Adding query filter params (name=service) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:54.573807 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35921272-e926-4725-b9fa-ec9b885fab37 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:54.575174 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35921272-e926-4725-b9fa-ec9b885fab37 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:54.583736 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 307/639] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Jun 16 17:30:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:54.584581 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35921272-e926-4725-b9fa-ec9b885fab37 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:54.610861 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57678aae-802a-471a-9726-d36a94ca264b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:54.611531 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57678aae-802a-471a-9726-d36a94ca264b None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:54.611815 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57678aae-802a-471a-9726-d36a94ca264b None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:54.612091 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57678aae-802a-471a-9726-d36a94ca264b None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:54.613284 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57678aae-802a-471a-9726-d36a94ca264b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:54.613616 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57678aae-802a-471a-9726-d36a94ca264b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:54.614960 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57678aae-802a-471a-9726-d36a94ca264b None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:54.627721 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 333/640] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jun 16 17:30:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:54.629699 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3e4375-012b-4b89-9522-acb92a8e03d4 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:54.630188 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3e4375-012b-4b89-9522-acb92a8e03d4 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:54.630661 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3e4375-012b-4b89-9522-acb92a8e03d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:54.648578 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-4d3e4375-012b-4b89-9522-acb92a8e03d4 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:30:54.649776 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-4d3e4375-012b-4b89-9522-acb92a8e03d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:30:54.662258 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:54.697473 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35921272-e926-4725-b9fa-ec9b885fab37 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:54.736526 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 334/641] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:30:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:30:54.736716 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3238aa56-f696-4c35-989d-829960981ac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:54.736867 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3e4375-012b-4b89-9522-acb92a8e03d4 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:54.736954 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3238aa56-f696-4c35-989d-829960981ac3 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:54.736954 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3238aa56-f696-4c35-989d-829960981ac3 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:54.736954 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3238aa56-f696-4c35-989d-829960981ac3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:54.746226 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3238aa56-f696-4c35-989d-829960981ac3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:54.747729 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3238aa56-f696-4c35-989d-829960981ac3 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:54.769066 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-3238aa56-f696-4c35-989d-829960981ac3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:30:54.769066 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:30:54.769066 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:30:54.769066 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:30:54.769066 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:30:54.769066 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:30:54.769066 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:30:54.769066 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:30:54.769066 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:30:54.769066 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:30:54.769066 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:30:54.769066 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:30:54.769066 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:30:54.769066 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:30:54.769066 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:30:54.770060 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:30:54.770060 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:30:54.770060 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:30:54.770060 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:30:54.770060 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:30:54.770060 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:30:54.770060 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:30:54.770060 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:30:54.770060 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:30:54.770060 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:30:54.770060 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:30:54.770060 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:30:54.770060 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:30:54.770060 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:30:54.770060 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:30:54.770060 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:30:54.770060 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:30:54.770929 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:30:54.770929 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:30:54.770929 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:30:54.770929 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:30:54.770929 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 308/642] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jun 16 17:30:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:30:54.771203 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3238aa56-f696-4c35-989d-829960981ac3 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:54.839606 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:54.861162 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3e4375-012b-4b89-9522-acb92a8e03d4 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:54.890347 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3238aa56-f696-4c35-989d-829960981ac3 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:54.902538 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6095c1ef-629b-4877-ad23-94878f3e77e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:54.903340 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6095c1ef-629b-4877-ad23-94878f3e77e4 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:54.903625 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6095c1ef-629b-4877-ad23-94878f3e77e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:54.903903 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6095c1ef-629b-4877-ad23-94878f3e77e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:54.915442 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6095c1ef-629b-4877-ad23-94878f3e77e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:54.916929 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6095c1ef-629b-4877-ad23-94878f3e77e4 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:54.917402 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:54.933691 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6317145b-3b12-4109-b9aa-15dabbddee86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:54.934490 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6317145b-3b12-4109-b9aa-15dabbddee86 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:54.934772 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6317145b-3b12-4109-b9aa-15dabbddee86 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:54.935047 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6317145b-3b12-4109-b9aa-15dabbddee86 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:54.936115 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-6095c1ef-629b-4877-ad23-94878f3e77e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:30:54.936115 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:30:54.936115 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:30:54.936115 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:30:54.936115 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:30:54.936115 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:30:54.936115 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:30:54.936115 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:30:54.936115 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:30:54.936115 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:30:54.936115 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:30:54.936115 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:30:54.936115 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:30:54.936115 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:30:54.936115 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:30:54.937916 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:30:54.937916 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:30:54.937916 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:30:54.937916 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:30:54.937916 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:30:54.937916 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:30:54.937916 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:30:54.937916 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:30:54.937916 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:30:54.937916 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:30:54.937916 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:30:54.937916 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:30:54.937916 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:30:54.937916 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:30:54.937916 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:30:54.937916 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:30:54.937916 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:30:54.938826 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:30:54.938826 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:30:54.938826 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:30:54.938826 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:30:54.938826 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 335/643] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jun 16 17:30:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:30:54.938826 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6095c1ef-629b-4877-ad23-94878f3e77e4 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:54.948352 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6317145b-3b12-4109-b9aa-15dabbddee86 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:54.949483 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6317145b-3b12-4109-b9aa-15dabbddee86 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:54.964943 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-6317145b-3b12-4109-b9aa-15dabbddee86 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:30:54.964943 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:30:54.964943 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:30:54.964943 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:30:54.964943 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:30:54.964943 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:30:54.964943 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:30:54.964943 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:30:54.964943 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:30:54.964943 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:30:54.964943 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:30:54.964943 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:30:54.964943 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:30:54.964943 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:30:54.964943 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:30:54.965897 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:30:54.965897 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:30:54.965897 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:30:54.965897 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:30:54.965897 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:30:54.965897 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:30:54.965897 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:30:54.965897 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:30:54.965897 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:30:54.965897 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:30:54.965897 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:30:54.965897 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:30:54.965897 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:30:54.965897 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:30:54.965897 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:30:54.965897 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:30:54.965897 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:30:54.966943 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:30:54.966943 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:30:54.966943 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:30:54.966943 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:30:54.966943 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:30:54.966943 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:30:54.966943 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:30:54.966943 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:30:54.966943 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 309/644] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jun 16 17:30:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:30:54.966943 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6317145b-3b12-4109-b9aa-15dabbddee86 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:55.052189 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6095c1ef-629b-4877-ad23-94878f3e77e4 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:55.073014 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6317145b-3b12-4109-b9aa-15dabbddee86 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:55.087415 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b83b82-7fa6-461d-9092-2bb2a125c297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:55.088212 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b83b82-7fa6-461d-9092-2bb2a125c297 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:55.088537 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b83b82-7fa6-461d-9092-2bb2a125c297 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:55.088820 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b83b82-7fa6-461d-9092-2bb2a125c297 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:55.095954 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:55.101190 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b83b82-7fa6-461d-9092-2bb2a125c297 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:30:55.109634 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-24b83b82-7fa6-461d-9092-2bb2a125c297 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:30:55.109634 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:30:55.109634 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:30:55.109634 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:30:55.109634 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:30:55.109634 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:30:55.109634 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:30:55.109634 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:30:55.109634 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:30:55.109634 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:30:55.109634 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:30:55.109634 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:30:55.109634 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:30:55.109634 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:30:55.109634 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:30:55.109634 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:30:55.110568 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:30:55.110568 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:30:55.110568 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:30:55.110568 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:30:55.110568 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:30:55.110568 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:30:55.110568 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:30:55.110568 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:30:55.110568 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:30:55.110568 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:30:55.110568 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:30:55.110568 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:30:55.110568 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:30:55.110568 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:30:55.110568 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:30:55.110568 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:30:55.111450 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:30:55.111450 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:30:55.111450 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:30:55.111450 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:30:55.111450 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:30:55.111450 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:30:55.111450 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:30:55.111450 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:30:55.111450 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:30:55.111450 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:30:55.111450 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:30:55.111450 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:30:55.111450 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 336/645] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Jun 16 17:30:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:30:55.112140 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b83b82-7fa6-461d-9092-2bb2a125c297 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:55.117639 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f26e58-770b-4bc4-bdce-fa339e6132d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:55.118544 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f26e58-770b-4bc4-bdce-fa339e6132d6 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:55.119004 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f26e58-770b-4bc4-bdce-fa339e6132d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:55.119454 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f26e58-770b-4bc4-bdce-fa339e6132d6 None admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:55.120352 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f26e58-770b-4bc4-bdce-fa339e6132d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:55.120859 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f26e58-770b-4bc4-bdce-fa339e6132d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:55.122432 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f26e58-770b-4bc4-bdce-fa339e6132d6 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:55.136659 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 310/646] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jun 16 17:30:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:55.137731 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f26e58-770b-4bc4-bdce-fa339e6132d6 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:55.198313 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:55.228232 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b83b82-7fa6-461d-9092-2bb2a125c297 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:55.262779 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f26e58-770b-4bc4-bdce-fa339e6132d6 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:55.266431 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdfa9d8-537d-4fc0-92f8-b8146c443ade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:55.267102 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdfa9d8-537d-4fc0-92f8-b8146c443ade None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:55.267385 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdfa9d8-537d-4fc0-92f8-b8146c443ade None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:55.267690 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdfa9d8-537d-4fc0-92f8-b8146c443ade None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:55.268400 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdfa9d8-537d-4fc0-92f8-b8146c443ade None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:55.268746 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdfa9d8-537d-4fc0-92f8-b8146c443ade None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:55.270117 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdfa9d8-537d-4fc0-92f8-b8146c443ade None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:55.290994 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 337/647] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jun 16 17:30:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:55.291664 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdfa9d8-537d-4fc0-92f8-b8146c443ade None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:55.300849 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7477a8-f5a1-47c0-85bb-c2d69f1980ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:55.301961 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7477a8-f5a1-47c0-85bb-c2d69f1980ce None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:55.302300 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7477a8-f5a1-47c0-85bb-c2d69f1980ce None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:55.302525 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7477a8-f5a1-47c0-85bb-c2d69f1980ce None admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:55.303682 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7477a8-f5a1-47c0-85bb-c2d69f1980ce None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:55.303993 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7477a8-f5a1-47c0-85bb-c2d69f1980ce None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:55.305791 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7477a8-f5a1-47c0-85bb-c2d69f1980ce None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:55.323662 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 311/648] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Jun 16 17:30:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:55.323662 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7477a8-f5a1-47c0-85bb-c2d69f1980ce None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:55.359047 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:55.411643 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdfa9d8-537d-4fc0-92f8-b8146c443ade None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:55.427974 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:55.434756 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7477a8-f5a1-47c0-85bb-c2d69f1980ce None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:55.456272 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5eb972-852a-411a-b6ca-e3e9ccf22f6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:55.456985 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5eb972-852a-411a-b6ca-e3e9ccf22f6d None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:55.457307 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5eb972-852a-411a-b6ca-e3e9ccf22f6d None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:55.457522 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5eb972-852a-411a-b6ca-e3e9ccf22f6d None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:55.458272 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5eb972-852a-411a-b6ca-e3e9ccf22f6d None admin] RBAC: Adding query filter params (name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:55.458578 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5eb972-852a-411a-b6ca-e3e9ccf22f6d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:55.459961 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5eb972-852a-411a-b6ca-e3e9ccf22f6d None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:55.468496 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 338/649] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Jun 16 17:30:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:55.469301 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5eb972-852a-411a-b6ca-e3e9ccf22f6d None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:55.482328 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f152c3-8b0b-46d4-bba9-896c05fd1742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:55.483225 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f152c3-8b0b-46d4-bba9-896c05fd1742 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:55.483610 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f152c3-8b0b-46d4-bba9-896c05fd1742 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:55.483859 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f152c3-8b0b-46d4-bba9-896c05fd1742 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:55.491530 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f152c3-8b0b-46d4-bba9-896c05fd1742 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:55.493005 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f152c3-8b0b-46d4-bba9-896c05fd1742 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:55.508314 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-a1f152c3-8b0b-46d4-bba9-896c05fd1742 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 17:30:55.508314 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:30:55.508314 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:30:55.508314 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:30:55.508314 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:30:55.508314 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:30:55.508314 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:30:55.508314 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:30:55.508314 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:30:55.508314 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:30:55.508314 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:30:55.508314 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:30:55.508314 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:30:55.508314 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:30:55.508314 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:30:55.508314 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:30:55.509344 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:30:55.509344 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:30:55.509344 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:30:55.509344 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:30:55.509344 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:30:55.509344 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:30:55.509344 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:30:55.509344 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:30:55.509344 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:30:55.509344 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:30:55.509344 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:30:55.509344 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:30:55.509344 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:30:55.509344 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:30:55.509344 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:30:55.509344 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:30:55.509344 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:30:55.509344 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:30:55.510286 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:30:55.510286 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:30:55.510286 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:30:55.510286 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:30:55.510286 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:30:55.510286 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:30:55.510286 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:30:55.510286 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:30:55.510286 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 17:30:55.510286 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:30:55.514147 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 312/650] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jun 16 17:30:55 2022] GET /identity/v3/users/cinder => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:30:55.515095 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f152c3-8b0b-46d4-bba9-896c05fd1742 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:55.580817 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5eb972-852a-411a-b6ca-e3e9ccf22f6d None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:55.602777 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:55.620692 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0bad3a-3242-47f7-8751-4cbb23582b3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:55.621620 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0bad3a-3242-47f7-8751-4cbb23582b3a None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:55.621971 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0bad3a-3242-47f7-8751-4cbb23582b3a None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:55.622321 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0bad3a-3242-47f7-8751-4cbb23582b3a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:55.630747 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0bad3a-3242-47f7-8751-4cbb23582b3a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:55.631466 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f152c3-8b0b-46d4-bba9-896c05fd1742 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:55.632194 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0bad3a-3242-47f7-8751-4cbb23582b3a None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:55.647664 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-fe0bad3a-3242-47f7-8751-4cbb23582b3a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 17:30:55.647664 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:30:55.647664 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:30:55.647664 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:30:55.647664 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:30:55.647664 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:30:55.647664 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:30:55.647664 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:30:55.647664 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:30:55.647664 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:30:55.647664 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:30:55.647664 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:30:55.647664 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:30:55.647664 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:30:55.647664 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:30:55.647664 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:30:55.648702 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:30:55.648702 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:30:55.648702 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:30:55.648702 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:30:55.648702 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:30:55.648702 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:30:55.648702 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:30:55.648702 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:30:55.648702 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:30:55.648702 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:30:55.648702 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:30:55.648702 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:30:55.648702 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:30:55.648702 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:30:55.648702 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:30:55.648702 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:30:55.648702 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:30:55.648702 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:30:55.649662 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:30:55.649662 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:30:55.649662 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:30:55.649662 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:30:55.649662 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:30:55.649662 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:30:55.649662 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:30:55.649662 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:30:55.649662 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 17:30:55.649662 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:30:55.649662 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 339/651] 10.222.0.42 () {56 vars in 1093 bytes} [Thu Jun 16 17:30:55 2022] GET /identity/v3/users/nova => generated 82 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:30:55.650244 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0bad3a-3242-47f7-8751-4cbb23582b3a None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:55.673527 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca373f96-79e8-40dd-8235-a952ab3afac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:55.674580 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca373f96-79e8-40dd-8235-a952ab3afac5 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:55.675033 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca373f96-79e8-40dd-8235-a952ab3afac5 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:55.675500 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca373f96-79e8-40dd-8235-a952ab3afac5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:55.676509 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca373f96-79e8-40dd-8235-a952ab3afac5 None admin] RBAC: Adding query filter params (role.id=cd7dd29fcdc844c78544402dc30986fb, scope.project.id=8abb87f2f141451a8c6ed832db82be18, user.id=d89029c1b3134df1962c041988bead74) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:55.677028 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca373f96-79e8-40dd-8235-a952ab3afac5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:55.678737 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca373f96-79e8-40dd-8235-a952ab3afac5 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:55.685761 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 313/652] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jun 16 17:30:55 2022] GET /identity/v3/role_assignments?user.id=d89029c1b3134df1962c041988bead74&scope.project.id=8abb87f2f141451a8c6ed832db82be18&role.id=cd7dd29fcdc844c78544402dc30986fb => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:55.686843 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca373f96-79e8-40dd-8235-a952ab3afac5 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:55.720848 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:55.796464 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0bad3a-3242-47f7-8751-4cbb23582b3a None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:55.826023 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca373f96-79e8-40dd-8235-a952ab3afac5 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:55.834595 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2575029-c085-43c0-b450-a1527bbb6655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:55.835754 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2575029-c085-43c0-b450-a1527bbb6655 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:55.836191 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2575029-c085-43c0-b450-a1527bbb6655 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:55.836700 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2575029-c085-43c0-b450-a1527bbb6655 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:55.850427 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2575029-c085-43c0-b450-a1527bbb6655 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:55.852311 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2575029-c085-43c0-b450-a1527bbb6655 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:55.872179 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b779a6e1-7ed9-4986-8dfa-331d88a8bbda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:55.872915 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b779a6e1-7ed9-4986-8dfa-331d88a8bbda None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:55.873199 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b779a6e1-7ed9-4986-8dfa-331d88a8bbda None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:55.873478 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b779a6e1-7ed9-4986-8dfa-331d88a8bbda None admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:55.874450 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b779a6e1-7ed9-4986-8dfa-331d88a8bbda None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:55.874835 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b779a6e1-7ed9-4986-8dfa-331d88a8bbda None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:55.875702 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-c2575029-c085-43c0-b450-a1527bbb6655 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:30:55.875702 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:30:55.875702 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:30:55.875702 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:30:55.875702 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:30:55.875702 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:30:55.875702 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:30:55.875702 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:30:55.875702 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:30:55.875702 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:30:55.875702 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:30:55.875702 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:30:55.875702 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:30:55.875702 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:30:55.875702 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:30:55.876798 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:30:55.876798 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:30:55.876798 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:30:55.876798 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:30:55.876798 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:30:55.876798 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:30:55.876798 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:30:55.876798 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:30:55.876798 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:30:55.876798 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:30:55.876798 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:30:55.876798 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:30:55.876798 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:30:55.876798 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:30:55.876798 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:30:55.876798 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:30:55.876798 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:30:55.877916 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:30:55.877916 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:30:55.877916 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:30:55.877916 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:30:55.878168 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b779a6e1-7ed9-4986-8dfa-331d88a8bbda None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:55.884849 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 340/653] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jun 16 17:30:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:30:55.886092 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2575029-c085-43c0-b450-a1527bbb6655 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:55.900417 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 314/654] 10.222.0.42 () {56 vars in 1137 bytes} [Thu Jun 16 17:30:55 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:55.901396 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b779a6e1-7ed9-4986-8dfa-331d88a8bbda None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:55.926696 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:56.009673 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:56.015357 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2575029-c085-43c0-b450-a1527bbb6655 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:56.027169 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b779a6e1-7ed9-4986-8dfa-331d88a8bbda None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:56.058978 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9d121b-20da-4e8d-9e0b-eef29cb3306e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:56.060310 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9d121b-20da-4e8d-9e0b-eef29cb3306e None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:56.061272 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9d121b-20da-4e8d-9e0b-eef29cb3306e None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:56.061760 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9d121b-20da-4e8d-9e0b-eef29cb3306e None admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:56.063419 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9d121b-20da-4e8d-9e0b-eef29cb3306e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:56.063912 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9d121b-20da-4e8d-9e0b-eef29cb3306e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:56.065960 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9d121b-20da-4e8d-9e0b-eef29cb3306e None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:56.070018 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c89bee-f689-4ea7-8a47-a804a08e4fb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:56.070786 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c89bee-f689-4ea7-8a47-a804a08e4fb1 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:56.071156 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c89bee-f689-4ea7-8a47-a804a08e4fb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:56.071491 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c89bee-f689-4ea7-8a47-a804a08e4fb1 None admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:56.072305 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c89bee-f689-4ea7-8a47-a804a08e4fb1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:56.072706 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c89bee-f689-4ea7-8a47-a804a08e4fb1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:56.074252 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c89bee-f689-4ea7-8a47-a804a08e4fb1 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:56.092456 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 315/655] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jun 16 17:30:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:56.092456 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c89bee-f689-4ea7-8a47-a804a08e4fb1 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:56.097701 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 341/656] 10.222.0.42 () {56 vars in 1133 bytes} [Thu Jun 16 17:30:55 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:56.102102 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b838ec8-ea70-44f6-8795-fc098a93cb05 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:56.102102 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b838ec8-ea70-44f6-8795-fc098a93cb05 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:56.102102 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b838ec8-ea70-44f6-8795-fc098a93cb05 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:56.104929 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 342/657] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:30:56 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:30:56.105792 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b838ec8-ea70-44f6-8795-fc098a93cb05 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:56.214633 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c89bee-f689-4ea7-8a47-a804a08e4fb1 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:56.221700 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:56.234808 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b838ec8-ea70-44f6-8795-fc098a93cb05 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:56.255133 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3748297-47a3-4242-9faa-d7c87ca64bb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:56.255852 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3748297-47a3-4242-9faa-d7c87ca64bb9 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:56.256170 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3748297-47a3-4242-9faa-d7c87ca64bb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:56.256526 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3748297-47a3-4242-9faa-d7c87ca64bb9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:56.271764 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:56.277721 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dcf1b0-133e-4f84-9578-79262159fcb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:56.278629 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dcf1b0-133e-4f84-9578-79262159fcb1 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:56.278943 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dcf1b0-133e-4f84-9578-79262159fcb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:56.279248 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dcf1b0-133e-4f84-9578-79262159fcb1 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:56.290159 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dcf1b0-133e-4f84-9578-79262159fcb1 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:56.292243 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dcf1b0-133e-4f84-9578-79262159fcb1 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:56.310046 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-a7dcf1b0-133e-4f84-9578-79262159fcb1 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 16 17:30:56.310046 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:30:56.310046 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:30:56.310046 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:30:56.310046 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:30:56.310046 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:30:56.310046 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:30:56.310046 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:30:56.310046 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:30:56.310046 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:30:56.310046 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:30:56.310046 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:30:56.310046 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:30:56.310046 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:30:56.310046 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:30:56.310046 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:30:56.312007 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:30:56.312007 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:30:56.312007 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:30:56.312007 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:30:56.312007 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:30:56.312007 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:30:56.312007 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:30:56.312007 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:30:56.312007 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:30:56.312007 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:30:56.312007 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:30:56.312007 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:30:56.312007 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:30:56.312007 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:30:56.312007 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:30:56.312007 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:30:56.312007 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:30:56.312007 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:30:56.314668 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:30:56.314668 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:30:56.314668 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:30:56.314668 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:30:56.314668 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:30:56.314668 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:30:56.314668 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:30:56.314668 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:30:56.314668 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 16 17:30:56.314668 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:30:56.314668 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 343/658] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Jun 16 17:30:56 2022] GET /identity/v3/users/neutron => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:30:56.314668 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dcf1b0-133e-4f84-9578-79262159fcb1 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:56.347129 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3748297-47a3-4242-9faa-d7c87ca64bb9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:56.348542 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3748297-47a3-4242-9faa-d7c87ca64bb9 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:56.429007 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dcf1b0-133e-4f84-9578-79262159fcb1 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:56.466828 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf487fa-7f0a-47b3-94e0-2f7f77696a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:56.467486 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf487fa-7f0a-47b3-94e0-2f7f77696a70 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:56.467773 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf487fa-7f0a-47b3-94e0-2f7f77696a70 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:56.468056 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf487fa-7f0a-47b3-94e0-2f7f77696a70 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:56.469076 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 316/659] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jun 16 17:30:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:30:56.470442 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b073bb90-1af1-4d5b-981c-140ba1343e4d None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:56.470758 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b073bb90-1af1-4d5b-981c-140ba1343e4d None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:56.471111 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b073bb90-1af1-4d5b-981c-140ba1343e4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:56.489968 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-b073bb90-1af1-4d5b-981c-140ba1343e4d None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:30:56.491223 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-b073bb90-1af1-4d5b-981c-140ba1343e4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:30:56.495516 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:56.537550 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:56.558677 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf487fa-7f0a-47b3-94e0-2f7f77696a70 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:56.560290 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf487fa-7f0a-47b3-94e0-2f7f77696a70 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:56.578889 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 317/660] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:30:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:30:56.580186 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b073bb90-1af1-4d5b-981c-140ba1343e4d None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:56.692034 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 344/661] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jun 16 17:30:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:30:56.693099 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf487fa-7f0a-47b3-94e0-2f7f77696a70 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:56.698592 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b073bb90-1af1-4d5b-981c-140ba1343e4d None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:56.738970 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b684e5d4-3709-4ff9-a3c7-096bd32c08d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:56.739966 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b684e5d4-3709-4ff9-a3c7-096bd32c08d7 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:56.740449 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b684e5d4-3709-4ff9-a3c7-096bd32c08d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:56.740738 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b684e5d4-3709-4ff9-a3c7-096bd32c08d7 None admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:56.741666 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b684e5d4-3709-4ff9-a3c7-096bd32c08d7 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:56.741980 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b684e5d4-3709-4ff9-a3c7-096bd32c08d7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:56.743447 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b684e5d4-3709-4ff9-a3c7-096bd32c08d7 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:56.748351 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:56.766290 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 318/662] 10.222.0.42 () {56 vars in 1139 bytes} [Thu Jun 16 17:30:56 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:56.767866 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b684e5d4-3709-4ff9-a3c7-096bd32c08d7 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:56.803245 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:56.806978 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf487fa-7f0a-47b3-94e0-2f7f77696a70 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:56.845272 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53187788-5e9d-43b5-af8d-37d748a27d2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:56.846065 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53187788-5e9d-43b5-af8d-37d748a27d2c None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:56.846350 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53187788-5e9d-43b5-af8d-37d748a27d2c None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:56.846624 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53187788-5e9d-43b5-af8d-37d748a27d2c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:56.856931 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53187788-5e9d-43b5-af8d-37d748a27d2c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:56.858385 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53187788-5e9d-43b5-af8d-37d748a27d2c None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:56.880728 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-53187788-5e9d-43b5-af8d-37d748a27d2c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:30:56.880728 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:30:56.880728 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:30:56.880728 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:30:56.880728 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:30:56.880728 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:30:56.880728 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:30:56.880728 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:30:56.880728 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:30:56.880728 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:30:56.880728 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:30:56.880728 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:30:56.880728 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:30:56.880728 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:30:56.880728 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:30:56.881684 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:30:56.881684 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:30:56.881684 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:30:56.881684 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:30:56.881684 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:30:56.881684 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:30:56.881684 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:30:56.881684 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:30:56.881684 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:30:56.881684 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:30:56.881684 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:30:56.881684 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:30:56.881684 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:30:56.881684 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:30:56.881684 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:30:56.881684 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:30:56.881684 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:30:56.882575 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:30:56.882575 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:30:56.882575 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:30:56.882575 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:30:56.886493 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 345/663] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jun 16 17:30:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:30:56.887206 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53187788-5e9d-43b5-af8d-37d748a27d2c None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:56.891723 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b684e5d4-3709-4ff9-a3c7-096bd32c08d7 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:56.929933 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566f529e-ab4f-46ad-a087-1a94d1558329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:56.930734 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566f529e-ab4f-46ad-a087-1a94d1558329 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:56.931017 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566f529e-ab4f-46ad-a087-1a94d1558329 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:56.931293 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566f529e-ab4f-46ad-a087-1a94d1558329 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:56.943895 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566f529e-ab4f-46ad-a087-1a94d1558329 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:30:56.951725 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-566f529e-ab4f-46ad-a087-1a94d1558329 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:30:56.951725 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:30:56.951725 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:30:56.951725 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:30:56.951725 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:30:56.951725 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:30:56.951725 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:30:56.951725 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:30:56.951725 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:30:56.951725 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:30:56.951725 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:30:56.951725 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:30:56.951725 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:30:56.951725 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:30:56.951725 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:30:56.951725 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:30:56.953450 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:30:56.953450 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:30:56.953450 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:30:56.953450 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:30:56.953450 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:30:56.953450 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:30:56.953450 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:30:56.953450 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:30:56.953450 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:30:56.953450 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:30:56.953450 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:30:56.953450 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:30:56.953450 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:30:56.953450 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:30:56.953450 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:30:56.953450 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:30:56.954321 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:30:56.954321 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:30:56.954321 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:30:56.954321 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:30:56.954321 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:30:56.954321 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:30:56.954321 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:30:56.954321 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:30:56.954321 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:30:56.954321 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:30:56.954321 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:30:56.954321 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:30:56.954321 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 319/664] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Jun 16 17:30:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:30:56.955551 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566f529e-ab4f-46ad-a087-1a94d1558329 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:57.007126 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53187788-5e9d-43b5-af8d-37d748a27d2c None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:57.008843 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:57.056171 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6df947-cb2a-45cf-9ceb-d22912e7caae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:57.057390 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6df947-cb2a-45cf-9ceb-d22912e7caae None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:57.057685 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6df947-cb2a-45cf-9ceb-d22912e7caae None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:57.057960 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6df947-cb2a-45cf-9ceb-d22912e7caae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:57.067165 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:57.071828 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6df947-cb2a-45cf-9ceb-d22912e7caae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:57.073333 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6df947-cb2a-45cf-9ceb-d22912e7caae None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:57.079453 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566f529e-ab4f-46ad-a087-1a94d1558329 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:57.092471 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-eb6df947-cb2a-45cf-9ceb-d22912e7caae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:30:57.092471 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:30:57.092471 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:30:57.092471 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:30:57.092471 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:30:57.092471 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:30:57.092471 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:30:57.092471 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:30:57.092471 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:30:57.092471 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:30:57.092471 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:30:57.092471 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:30:57.092471 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:30:57.092471 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:30:57.092471 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:30:57.093451 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:30:57.093451 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:30:57.093451 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:30:57.093451 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:30:57.093451 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:30:57.093451 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:30:57.093451 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:30:57.093451 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:30:57.093451 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:30:57.093451 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:30:57.093451 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:30:57.093451 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:30:57.093451 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:30:57.093451 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:30:57.093451 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:30:57.093451 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:30:57.093451 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:30:57.094479 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:30:57.094479 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:30:57.094479 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:30:57.094479 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:30:57.094479 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 346/665] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jun 16 17:30:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:30:57.094763 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6df947-cb2a-45cf-9ceb-d22912e7caae None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:57.117048 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24378a9d-bbb9-4a26-8ca1-bb8060c1c606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:57.117701 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24378a9d-bbb9-4a26-8ca1-bb8060c1c606 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:57.117986 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24378a9d-bbb9-4a26-8ca1-bb8060c1c606 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:57.118386 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24378a9d-bbb9-4a26-8ca1-bb8060c1c606 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:57.194955 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24378a9d-bbb9-4a26-8ca1-bb8060c1c606 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:57.196640 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24378a9d-bbb9-4a26-8ca1-bb8060c1c606 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:57.204605 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6df947-cb2a-45cf-9ceb-d22912e7caae None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:57.242534 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56205329-697d-4295-8338-8c2f36c6f3f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:57.243202 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56205329-697d-4295-8338-8c2f36c6f3f8 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:57.243487 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56205329-697d-4295-8338-8c2f36c6f3f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:57.243763 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56205329-697d-4295-8338-8c2f36c6f3f8 None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:57.244516 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56205329-697d-4295-8338-8c2f36c6f3f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:57.244827 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56205329-697d-4295-8338-8c2f36c6f3f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:57.246281 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56205329-697d-4295-8338-8c2f36c6f3f8 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:57.259662 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 347/666] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jun 16 17:30:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:57.260468 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56205329-697d-4295-8338-8c2f36c6f3f8 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:57.277356 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:57.314885 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:57.325557 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 320/667] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jun 16 17:30:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:30:57.326506 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24378a9d-bbb9-4a26-8ca1-bb8060c1c606 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:57.383718 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56205329-697d-4295-8338-8c2f36c6f3f8 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:57.420555 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5c4c4f-b60b-4a53-9f2a-84ab146683dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:57.421357 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5c4c4f-b60b-4a53-9f2a-84ab146683dc None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:57.422209 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5c4c4f-b60b-4a53-9f2a-84ab146683dc None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:57.422608 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5c4c4f-b60b-4a53-9f2a-84ab146683dc None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:57.423356 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5c4c4f-b60b-4a53-9f2a-84ab146683dc None admin] RBAC: Adding query filter params (name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:57.423670 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5c4c4f-b60b-4a53-9f2a-84ab146683dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:57.425198 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5c4c4f-b60b-4a53-9f2a-84ab146683dc None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:57.431219 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24378a9d-bbb9-4a26-8ca1-bb8060c1c606 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:57.433962 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 348/668] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Jun 16 17:30:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:57.434772 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5c4c4f-b60b-4a53-9f2a-84ab146683dc None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:57.473113 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3d7224-2484-4d79-9221-7863bb0bb5b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:57.473716 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3d7224-2484-4d79-9221-7863bb0bb5b0 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:57.474001 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3d7224-2484-4d79-9221-7863bb0bb5b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:57.474278 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3d7224-2484-4d79-9221-7863bb0bb5b0 None admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:57.475003 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3d7224-2484-4d79-9221-7863bb0bb5b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:57.475310 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3d7224-2484-4d79-9221-7863bb0bb5b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:57.476719 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3d7224-2484-4d79-9221-7863bb0bb5b0 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:57.494256 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 321/669] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jun 16 17:30:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:57.494256 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3d7224-2484-4d79-9221-7863bb0bb5b0 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:57.523553 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:57.545086 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:57.553592 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5c4c4f-b60b-4a53-9f2a-84ab146683dc None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:57.588527 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb385826-6fcf-456d-8f22-5ce63c670b5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:57.589343 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb385826-6fcf-456d-8f22-5ce63c670b5b None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:57.589623 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb385826-6fcf-456d-8f22-5ce63c670b5b None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:57.589904 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb385826-6fcf-456d-8f22-5ce63c670b5b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:57.598042 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb385826-6fcf-456d-8f22-5ce63c670b5b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:57.599509 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb385826-6fcf-456d-8f22-5ce63c670b5b None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:57.602379 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3d7224-2484-4d79-9221-7863bb0bb5b0 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:57.614700 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-eb385826-6fcf-456d-8f22-5ce63c670b5b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:30:57.614700 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:30:57.614700 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:30:57.614700 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:30:57.614700 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:30:57.614700 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:30:57.614700 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:30:57.614700 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:30:57.614700 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:30:57.614700 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:30:57.614700 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:30:57.614700 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:30:57.614700 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:30:57.614700 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:30:57.614700 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:30:57.615619 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:30:57.615619 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:30:57.615619 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:30:57.615619 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:30:57.615619 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:30:57.615619 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:30:57.615619 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:30:57.615619 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:30:57.615619 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:30:57.615619 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:30:57.615619 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:30:57.615619 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:30:57.615619 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:30:57.615619 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:30:57.615619 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:30:57.615619 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:30:57.615619 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:30:57.616570 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:30:57.616570 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:30:57.616570 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:30:57.616570 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:30:57.616570 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:30:57.616570 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:30:57.616570 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:30:57.616570 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:30:57.616570 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 349/670] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jun 16 17:30:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:30:57.617063 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb385826-6fcf-456d-8f22-5ce63c670b5b None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:57.645127 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c2d513-f3c5-4940-a758-bb866d784ac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:57.645955 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c2d513-f3c5-4940-a758-bb866d784ac1 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:57.646239 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c2d513-f3c5-4940-a758-bb866d784ac1 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:57.646510 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c2d513-f3c5-4940-a758-bb866d784ac1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:57.659167 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c2d513-f3c5-4940-a758-bb866d784ac1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:57.660643 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c2d513-f3c5-4940-a758-bb866d784ac1 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:57.679192 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-62c2d513-f3c5-4940-a758-bb866d784ac1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:30:57.679192 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:30:57.679192 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:30:57.679192 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:30:57.679192 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:30:57.679192 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:30:57.679192 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:30:57.679192 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:30:57.679192 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:30:57.679192 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:30:57.679192 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:30:57.679192 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:30:57.679192 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:30:57.679192 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:30:57.679192 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:30:57.680357 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:30:57.680357 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:30:57.680357 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:30:57.680357 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:30:57.680357 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:30:57.680357 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:30:57.680357 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:30:57.680357 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:30:57.680357 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:30:57.680357 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:30:57.680357 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:30:57.680357 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:30:57.680357 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:30:57.680357 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:30:57.680357 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:30:57.680357 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:30:57.680357 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:30:57.681309 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:30:57.681309 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:30:57.681309 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:30:57.681309 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:30:57.681309 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 322/671] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jun 16 17:30:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:30:57.681593 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c2d513-f3c5-4940-a758-bb866d784ac1 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:57.744306 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb385826-6fcf-456d-8f22-5ce63c670b5b None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:57.779987 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1797bd-c1d5-4c93-a19f-6ae66fd0b4c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:57.781394 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1797bd-c1d5-4c93-a19f-6ae66fd0b4c2 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:57.781691 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1797bd-c1d5-4c93-a19f-6ae66fd0b4c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:57.782119 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1797bd-c1d5-4c93-a19f-6ae66fd0b4c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:57.785511 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:57.795050 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1797bd-c1d5-4c93-a19f-6ae66fd0b4c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:57.797184 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1797bd-c1d5-4c93-a19f-6ae66fd0b4c2 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:57.803617 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c2d513-f3c5-4940-a758-bb866d784ac1 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:57.805328 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:57.817000 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-2c1797bd-c1d5-4c93-a19f-6ae66fd0b4c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:30:57.817000 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:30:57.817000 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:30:57.817000 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:30:57.817000 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:30:57.817000 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:30:57.817000 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:30:57.817000 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:30:57.817000 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:30:57.817000 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:30:57.817000 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:30:57.817000 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:30:57.817000 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:30:57.817000 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:30:57.817000 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:30:57.817917 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:30:57.817917 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:30:57.817917 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:30:57.817917 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:30:57.817917 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:30:57.817917 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:30:57.817917 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:30:57.817917 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:30:57.817917 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:30:57.817917 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:30:57.817917 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:30:57.817917 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:30:57.817917 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:30:57.817917 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:30:57.817917 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:30:57.817917 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:30:57.817917 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:30:57.818980 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:30:57.818980 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:30:57.818980 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:30:57.818980 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:30:57.818980 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 350/672] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jun 16 17:30:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:30:57.819254 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1797bd-c1d5-4c93-a19f-6ae66fd0b4c2 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:57.842393 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f79f8a8-76e2-46bd-b005-ba394da84af0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:57.843194 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f79f8a8-76e2-46bd-b005-ba394da84af0 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:57.844004 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f79f8a8-76e2-46bd-b005-ba394da84af0 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:57.844710 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f79f8a8-76e2-46bd-b005-ba394da84af0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:57.851752 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f79f8a8-76e2-46bd-b005-ba394da84af0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:57.853138 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f79f8a8-76e2-46bd-b005-ba394da84af0 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:57.877633 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-5f79f8a8-76e2-46bd-b005-ba394da84af0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:30:57.877633 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:30:57.877633 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:30:57.877633 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:30:57.877633 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:30:57.877633 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:30:57.877633 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:30:57.877633 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:30:57.877633 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:30:57.877633 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:30:57.877633 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:30:57.877633 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:30:57.877633 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:30:57.877633 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:30:57.877633 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:30:57.878586 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:30:57.878586 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:30:57.878586 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:30:57.878586 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:30:57.878586 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:30:57.878586 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:30:57.878586 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:30:57.878586 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:30:57.878586 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:30:57.878586 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:30:57.878586 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:30:57.878586 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:30:57.878586 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:30:57.878586 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:30:57.878586 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:30:57.878586 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:30:57.878586 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:30:57.879478 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:30:57.879478 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:30:57.879478 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:30:57.879478 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:30:57.879478 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:30:57.879478 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:30:57.879478 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:30:57.879478 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:30:57.879478 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 323/673] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jun 16 17:30:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:30:57.880645 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f79f8a8-76e2-46bd-b005-ba394da84af0 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:57.947614 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1797bd-c1d5-4c93-a19f-6ae66fd0b4c2 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:57.983871 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab5ab0c-4a0f-48ae-bfcd-2ba4d9c0475d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:57.985156 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab5ab0c-4a0f-48ae-bfcd-2ba4d9c0475d None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:57.985581 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab5ab0c-4a0f-48ae-bfcd-2ba4d9c0475d None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:57.985860 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab5ab0c-4a0f-48ae-bfcd-2ba4d9c0475d None admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:57.986727 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab5ab0c-4a0f-48ae-bfcd-2ba4d9c0475d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:57.987026 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab5ab0c-4a0f-48ae-bfcd-2ba4d9c0475d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:57.988531 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab5ab0c-4a0f-48ae-bfcd-2ba4d9c0475d None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:57.994269 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f79f8a8-76e2-46bd-b005-ba394da84af0 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:58.009713 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 351/674] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Jun 16 17:30:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:58.009713 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab5ab0c-4a0f-48ae-bfcd-2ba4d9c0475d None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:58.044381 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4dace7-64ec-4390-abcd-ec1344ebf90f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:58.045205 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4dace7-64ec-4390-abcd-ec1344ebf90f None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:58.045558 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4dace7-64ec-4390-abcd-ec1344ebf90f None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:58.045906 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4dace7-64ec-4390-abcd-ec1344ebf90f None admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:58.047236 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4dace7-64ec-4390-abcd-ec1344ebf90f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:58.047613 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4dace7-64ec-4390-abcd-ec1344ebf90f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:58.049116 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4dace7-64ec-4390-abcd-ec1344ebf90f None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:58.061149 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 324/675] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jun 16 17:30:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:58.062012 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4dace7-64ec-4390-abcd-ec1344ebf90f None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:58.072720 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:58.090322 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:58.129141 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab5ab0c-4a0f-48ae-bfcd-2ba4d9c0475d None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:58.165687 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7da633-29de-4e0d-9f7a-b77ee4cdf50b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:58.165981 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4dace7-64ec-4390-abcd-ec1344ebf90f None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:58.166394 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7da633-29de-4e0d-9f7a-b77ee4cdf50b None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:58.166708 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7da633-29de-4e0d-9f7a-b77ee4cdf50b None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:58.166979 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7da633-29de-4e0d-9f7a-b77ee4cdf50b None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:58.167741 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7da633-29de-4e0d-9f7a-b77ee4cdf50b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:58.168078 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7da633-29de-4e0d-9f7a-b77ee4cdf50b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:58.169491 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7da633-29de-4e0d-9f7a-b77ee4cdf50b None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:58.182212 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 352/676] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jun 16 17:30:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:58.183052 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7da633-29de-4e0d-9f7a-b77ee4cdf50b None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:58.204724 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4eb570-4b63-42e1-ac95-1c440cdff6fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:58.205795 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4eb570-4b63-42e1-ac95-1c440cdff6fc None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:58.206339 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4eb570-4b63-42e1-ac95-1c440cdff6fc None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:58.206694 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4eb570-4b63-42e1-ac95-1c440cdff6fc None admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:58.207254 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4eb570-4b63-42e1-ac95-1c440cdff6fc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:58.207602 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4eb570-4b63-42e1-ac95-1c440cdff6fc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:58.209163 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4eb570-4b63-42e1-ac95-1c440cdff6fc None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:58.222117 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 325/677] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Jun 16 17:30:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:58.222938 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4eb570-4b63-42e1-ac95-1c440cdff6fc None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:58.289804 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7da633-29de-4e0d-9f7a-b77ee4cdf50b None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:58.299547 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:58.307495 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:58.325527 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57228258-89d9-4037-b571-0a2d053e7924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:58.326187 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57228258-89d9-4037-b571-0a2d053e7924 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:58.326350 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4eb570-4b63-42e1-ac95-1c440cdff6fc None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:58.326475 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57228258-89d9-4037-b571-0a2d053e7924 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:58.326738 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57228258-89d9-4037-b571-0a2d053e7924 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:58.327623 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57228258-89d9-4037-b571-0a2d053e7924 None admin] RBAC: Adding query filter params (role.id=cd7dd29fcdc844c78544402dc30986fb, scope.project.id=8abb87f2f141451a8c6ed832db82be18, user.id=24d5e2637b7043e08313909d5b06e329) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:58.328680 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57228258-89d9-4037-b571-0a2d053e7924 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:58.330187 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57228258-89d9-4037-b571-0a2d053e7924 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:58.336314 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 353/678] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jun 16 17:30:58 2022] GET /identity/v3/role_assignments?user.id=24d5e2637b7043e08313909d5b06e329&scope.project.id=8abb87f2f141451a8c6ed832db82be18&role.id=cd7dd29fcdc844c78544402dc30986fb => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:58.337715 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a0c23e-17af-4ce1-9946-869f82378410 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:58.338025 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a0c23e-17af-4ce1-9946-869f82378410 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:58.338327 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a0c23e-17af-4ce1-9946-869f82378410 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:58.339659 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 354/679] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:30:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:30:58.340417 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a0c23e-17af-4ce1-9946-869f82378410 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:58.377393 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5106d50a-b398-4ea1-aff7-7ff42cd43115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:58.378846 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5106d50a-b398-4ea1-aff7-7ff42cd43115 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:58.379400 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5106d50a-b398-4ea1-aff7-7ff42cd43115 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:58.379678 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5106d50a-b398-4ea1-aff7-7ff42cd43115 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:58.390133 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5106d50a-b398-4ea1-aff7-7ff42cd43115 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:58.392947 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5106d50a-b398-4ea1-aff7-7ff42cd43115 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:58.408026 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-5106d50a-b398-4ea1-aff7-7ff42cd43115 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:30:58.408026 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:30:58.408026 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:30:58.408026 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:30:58.408026 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:30:58.408026 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:30:58.408026 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:30:58.408026 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:30:58.408026 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:30:58.408026 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:30:58.408026 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:30:58.408026 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:30:58.408026 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:30:58.408026 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:30:58.408026 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:30:58.409270 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:30:58.409270 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:30:58.409270 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:30:58.409270 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:30:58.409270 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:30:58.409270 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:30:58.409270 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:30:58.409270 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:30:58.409270 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:30:58.409270 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:30:58.409270 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:30:58.409270 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:30:58.409270 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:30:58.409270 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:30:58.409270 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:30:58.409270 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:30:58.409270 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:30:58.410158 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:30:58.410158 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:30:58.410158 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:30:58.410158 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:30:58.410158 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:30:58.410158 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:30:58.410158 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:30:58.410158 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:30:58.410158 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 326/680] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jun 16 17:30:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:30:58.410657 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5106d50a-b398-4ea1-aff7-7ff42cd43115 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:58.472338 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a0c23e-17af-4ce1-9946-869f82378410 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:58.515639 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a7e6f8-96d5-4a82-a40b-42c7442ba36e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:58.516481 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a7e6f8-96d5-4a82-a40b-42c7442ba36e None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:58.516764 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a7e6f8-96d5-4a82-a40b-42c7442ba36e None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:58.517043 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a7e6f8-96d5-4a82-a40b-42c7442ba36e None admin] PATH_INFO: `/v3/users/swift` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:58.526260 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a7e6f8-96d5-4a82-a40b-42c7442ba36e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:58.527792 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a7e6f8-96d5-4a82-a40b-42c7442ba36e None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:58.534535 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5106d50a-b398-4ea1-aff7-7ff42cd43115 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:58.544152 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-60a7e6f8-96d5-4a82-a40b-42c7442ba36e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 16 17:30:58.544152 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:30:58.544152 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:30:58.544152 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:30:58.544152 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:30:58.544152 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:30:58.544152 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:30:58.544152 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:30:58.544152 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:30:58.544152 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:30:58.544152 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:30:58.544152 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:30:58.544152 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:30:58.544152 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:30:58.544152 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:30:58.544152 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:30:58.545442 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:30:58.545442 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:30:58.545442 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:30:58.545442 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:30:58.545442 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:30:58.545442 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:30:58.545442 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:30:58.545442 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:30:58.545442 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:30:58.545442 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:30:58.545442 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:30:58.545442 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:30:58.545442 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:30:58.545442 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:30:58.545442 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:30:58.545442 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:30:58.545442 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:30:58.545442 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:30:58.546626 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:30:58.546626 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:30:58.546626 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:30:58.546626 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:30:58.546626 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:30:58.546626 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:30:58.546626 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:30:58.546626 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:30:58.546626 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 16 17:30:58.546626 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:30:58.546626 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 355/681] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Jun 16 17:30:58 2022] GET /identity/v3/users/swift => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:30:58.548009 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf145b60-31fc-4523-aa62-615d9b7cf0aa None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:58.548302 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf145b60-31fc-4523-aa62-615d9b7cf0aa None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:58.548615 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf145b60-31fc-4523-aa62-615d9b7cf0aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:58.569872 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-cf145b60-31fc-4523-aa62-615d9b7cf0aa None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:30:58.571055 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-cf145b60-31fc-4523-aa62-615d9b7cf0aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:30:58.578909 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74277cb7-dd2a-46cc-a459-9b06dc35280a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:58.580050 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74277cb7-dd2a-46cc-a459-9b06dc35280a None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:58.580340 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74277cb7-dd2a-46cc-a459-9b06dc35280a None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:58.580646 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74277cb7-dd2a-46cc-a459-9b06dc35280a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:58.581854 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74277cb7-dd2a-46cc-a459-9b06dc35280a None admin] RBAC: Adding query filter params (role.id=cd7dd29fcdc844c78544402dc30986fb, scope.project.id=8abb87f2f141451a8c6ed832db82be18, user.id=f86c16179a70474a817be9fda33ece9d) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:58.582174 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74277cb7-dd2a-46cc-a459-9b06dc35280a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:58.583945 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74277cb7-dd2a-46cc-a459-9b06dc35280a None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:58.600402 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 327/682] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jun 16 17:30:58 2022] GET /identity/v3/role_assignments?user.id=f86c16179a70474a817be9fda33ece9d&scope.project.id=8abb87f2f141451a8c6ed832db82be18&role.id=cd7dd29fcdc844c78544402dc30986fb => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:58.600402 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74277cb7-dd2a-46cc-a459-9b06dc35280a None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:58.600683 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:58.616216 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:58.653279 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 356/683] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:30:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:30:58.654420 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf145b60-31fc-4523-aa62-615d9b7cf0aa None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:58.772322 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74277cb7-dd2a-46cc-a459-9b06dc35280a None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:58.793283 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf145b60-31fc-4523-aa62-615d9b7cf0aa None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:58.819086 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ffdef1-8219-484b-a924-e98e94f759f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:58.820070 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ffdef1-8219-484b-a924-e98e94f759f2 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:58.820070 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ffdef1-8219-484b-a924-e98e94f759f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:58.820358 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ffdef1-8219-484b-a924-e98e94f759f2 None admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:58.821015 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ffdef1-8219-484b-a924-e98e94f759f2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:58.821322 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ffdef1-8219-484b-a924-e98e94f759f2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:58.822774 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ffdef1-8219-484b-a924-e98e94f759f2 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:58.837900 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bb7db7-f6fb-49b8-bf75-7193eb1c7c81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:58.838128 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 328/684] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Jun 16 17:30:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:58.838583 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bb7db7-f6fb-49b8-bf75-7193eb1c7c81 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:58.838869 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bb7db7-f6fb-49b8-bf75-7193eb1c7c81 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:58.838969 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ffdef1-8219-484b-a924-e98e94f759f2 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:58.839170 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bb7db7-f6fb-49b8-bf75-7193eb1c7c81 None admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:58.839979 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bb7db7-f6fb-49b8-bf75-7193eb1c7c81 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:58.840290 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bb7db7-f6fb-49b8-bf75-7193eb1c7c81 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:58.841799 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bb7db7-f6fb-49b8-bf75-7193eb1c7c81 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:58.870767 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 357/685] 10.222.0.42 () {56 vars in 1135 bytes} [Thu Jun 16 17:30:58 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:58.873184 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bb7db7-f6fb-49b8-bf75-7193eb1c7c81 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:58.903851 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:58.930897 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:58.978418 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ffdef1-8219-484b-a924-e98e94f759f2 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:59.017008 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52dce9f-bf6f-4e93-896a-048dfc2b2492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:59.017827 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52dce9f-bf6f-4e93-896a-048dfc2b2492 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:59.018165 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52dce9f-bf6f-4e93-896a-048dfc2b2492 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:59.018423 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52dce9f-bf6f-4e93-896a-048dfc2b2492 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:59.020661 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bb7db7-f6fb-49b8-bf75-7193eb1c7c81 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:59.029548 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52dce9f-bf6f-4e93-896a-048dfc2b2492 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:59.031789 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52dce9f-bf6f-4e93-896a-048dfc2b2492 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:59.051208 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-a52dce9f-bf6f-4e93-896a-048dfc2b2492 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:30:59.051208 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:30:59.051208 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:30:59.051208 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:30:59.051208 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:30:59.051208 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:30:59.051208 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:30:59.051208 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:30:59.051208 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:30:59.051208 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:30:59.051208 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:30:59.051208 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:30:59.051208 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:30:59.051208 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:30:59.051208 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:30:59.052172 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:30:59.052172 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:30:59.052172 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:30:59.052172 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:30:59.052172 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:30:59.052172 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:30:59.052172 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:30:59.052172 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:30:59.052172 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:30:59.052172 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:30:59.052172 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:30:59.052172 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:30:59.052172 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:30:59.052172 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:30:59.052172 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:30:59.052172 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:30:59.052172 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:30:59.053161 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:30:59.053161 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:30:59.053161 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:30:59.053161 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:30:59.053161 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 329/686] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jun 16 17:30:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:30:59.058653 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52dce9f-bf6f-4e93-896a-048dfc2b2492 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:59.076651 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23e3581-6888-40c1-b7ba-9c34df630107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:59.078252 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23e3581-6888-40c1-b7ba-9c34df630107 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:59.078920 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23e3581-6888-40c1-b7ba-9c34df630107 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:59.079980 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23e3581-6888-40c1-b7ba-9c34df630107 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:59.081378 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23e3581-6888-40c1-b7ba-9c34df630107 None admin] RBAC: Adding query filter params (role.id=cd7dd29fcdc844c78544402dc30986fb, scope.project.id=8abb87f2f141451a8c6ed832db82be18, user.id=f94416103f6245888214a997b014f47a) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:59.081920 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23e3581-6888-40c1-b7ba-9c34df630107 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:59.084470 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23e3581-6888-40c1-b7ba-9c34df630107 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:59.100411 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 358/687] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jun 16 17:30:58 2022] GET /identity/v3/role_assignments?user.id=f94416103f6245888214a997b014f47a&scope.project.id=8abb87f2f141451a8c6ed832db82be18&role.id=cd7dd29fcdc844c78544402dc30986fb => generated 608 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:59.100411 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23e3581-6888-40c1-b7ba-9c34df630107 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:59.214589 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:59.254469 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23e3581-6888-40c1-b7ba-9c34df630107 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:59.259579 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52dce9f-bf6f-4e93-896a-048dfc2b2492 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:59.306458 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:59.336981 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc30693-2e7f-47ad-8bc5-3fd1dee127e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:59.337567 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc30693-2e7f-47ad-8bc5-3fd1dee127e2 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:59.337852 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc30693-2e7f-47ad-8bc5-3fd1dee127e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:59.338130 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc30693-2e7f-47ad-8bc5-3fd1dee127e2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:59.345194 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95be14d-c832-48f2-a0df-13b27d1e0f8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:59.345884 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95be14d-c832-48f2-a0df-13b27d1e0f8b None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:59.346170 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95be14d-c832-48f2-a0df-13b27d1e0f8b None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:59.346452 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95be14d-c832-48f2-a0df-13b27d1e0f8b None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:59.347186 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95be14d-c832-48f2-a0df-13b27d1e0f8b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:59.347497 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95be14d-c832-48f2-a0df-13b27d1e0f8b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:59.348926 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95be14d-c832-48f2-a0df-13b27d1e0f8b None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:59.376914 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 359/688] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jun 16 17:30:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:59.385606 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95be14d-c832-48f2-a0df-13b27d1e0f8b None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:59.425475 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc30693-2e7f-47ad-8bc5-3fd1dee127e2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:59.428031 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc30693-2e7f-47ad-8bc5-3fd1dee127e2 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:59.540789 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95be14d-c832-48f2-a0df-13b27d1e0f8b None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:59.560313 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 330/689] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jun 16 17:30:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 507 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:30:59.568294 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc30693-2e7f-47ad-8bc5-3fd1dee127e2 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:59.621012 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:59.623152 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cfde21-9935-456d-afcf-769ef2bf9451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:59.624749 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cfde21-9935-456d-afcf-769ef2bf9451 None admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:59.625218 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cfde21-9935-456d-afcf-769ef2bf9451 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:59.625707 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cfde21-9935-456d-afcf-769ef2bf9451 None admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:59.626480 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cfde21-9935-456d-afcf-769ef2bf9451 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:59.627924 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cfde21-9935-456d-afcf-769ef2bf9451 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:59.656824 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:59.663883 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 360/690] 10.222.0.42 () {60 vars in 1132 bytes} [Thu Jun 16 17:30:59 2022] POST /identity/v3/users => generated 254 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:30:59.748524 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc30693-2e7f-47ad-8bc5-3fd1dee127e2 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:59.788463 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f22364b-0617-400c-bdfd-6490c7cd64cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:59.789276 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f22364b-0617-400c-bdfd-6490c7cd64cb None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:59.789555 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f22364b-0617-400c-bdfd-6490c7cd64cb None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:59.789828 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f22364b-0617-400c-bdfd-6490c7cd64cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:59.809481 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f22364b-0617-400c-bdfd-6490c7cd64cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:59.810992 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f22364b-0617-400c-bdfd-6490c7cd64cb None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:59.838058 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-8f22364b-0617-400c-bdfd-6490c7cd64cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:30:59.838058 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:30:59.838058 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:30:59.838058 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:30:59.838058 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:30:59.838058 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:30:59.838058 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:30:59.838058 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:30:59.838058 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:30:59.838058 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:30:59.838058 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:30:59.838058 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:30:59.838058 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:30:59.838058 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:30:59.838058 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:30:59.840231 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:30:59.840231 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:30:59.840231 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:30:59.840231 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:30:59.840231 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:30:59.840231 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:30:59.840231 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:30:59.840231 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:30:59.840231 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:30:59.840231 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:30:59.840231 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:30:59.840231 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:30:59.840231 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:30:59.840231 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:30:59.840231 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:30:59.840231 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:30:59.840231 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:30:59.842408 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:30:59.842408 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:30:59.842408 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:30:59.842408 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:30:59.842408 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 331/691] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jun 16 17:30:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:30:59.849438 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cfde21-9935-456d-afcf-769ef2bf9451 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:00.022056 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cfde21-9935-456d-afcf-769ef2bf9451 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:00.069406 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9577fc9e-46ac-4248-a972-e3e192a6fecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:00.070150 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9577fc9e-46ac-4248-a972-e3e192a6fecc None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:00.070436 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9577fc9e-46ac-4248-a972-e3e192a6fecc None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:00.070716 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9577fc9e-46ac-4248-a972-e3e192a6fecc None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:00.071458 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9577fc9e-46ac-4248-a972-e3e192a6fecc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:00.071827 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9577fc9e-46ac-4248-a972-e3e192a6fecc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:00.073281 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9577fc9e-46ac-4248-a972-e3e192a6fecc None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:00.088356 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 361/692] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jun 16 17:30:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:00.094185 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f22364b-0617-400c-bdfd-6490c7cd64cb None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:00.175740 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:00.222781 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f22364b-0617-400c-bdfd-6490c7cd64cb None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:00.268555 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d91444-186b-4ce0-8a8e-5dd84577b525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:00.269364 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d91444-186b-4ce0-8a8e-5dd84577b525 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:00.269647 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d91444-186b-4ce0-8a8e-5dd84577b525 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:00.269925 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d91444-186b-4ce0-8a8e-5dd84577b525 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:00.277397 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d91444-186b-4ce0-8a8e-5dd84577b525 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:00.278855 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d91444-186b-4ce0-8a8e-5dd84577b525 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:00.295655 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-42d91444-186b-4ce0-8a8e-5dd84577b525 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:31:00.295655 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:00.295655 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:31:00.295655 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:31:00.295655 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:31:00.295655 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:31:00.295655 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:31:00.295655 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:31:00.295655 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:31:00.295655 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:31:00.295655 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:31:00.295655 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:31:00.295655 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:31:00.295655 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:31:00.295655 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:31:00.296648 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:31:00.296648 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:31:00.296648 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:31:00.296648 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:31:00.296648 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:31:00.296648 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:31:00.296648 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:31:00.296648 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:31:00.296648 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:31:00.296648 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:31:00.296648 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:31:00.296648 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:31:00.296648 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:31:00.296648 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:31:00.296648 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:31:00.296648 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:31:00.296648 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:31:00.298152 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:31:00.298152 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:31:00.298152 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:31:00.298152 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:31:00.298152 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:31:00.298152 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:31:00.298152 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:31:00.298152 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:31:00.298592 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 332/693] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jun 16 17:31:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:31:00.303711 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9577fc9e-46ac-4248-a972-e3e192a6fecc None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:00.368300 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:00.413450 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9577fc9e-46ac-4248-a972-e3e192a6fecc None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:00.450007 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba05b02-c75f-41e0-b977-49c511e7a067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:00.450644 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba05b02-c75f-41e0-b977-49c511e7a067 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:00.450932 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba05b02-c75f-41e0-b977-49c511e7a067 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:00.451210 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba05b02-c75f-41e0-b977-49c511e7a067 None admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:00.451994 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba05b02-c75f-41e0-b977-49c511e7a067 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:00.452304 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba05b02-c75f-41e0-b977-49c511e7a067 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:00.455606 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba05b02-c75f-41e0-b977-49c511e7a067 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:00.468047 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 362/694] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Jun 16 17:31:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:31:00.477513 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d91444-186b-4ce0-8a8e-5dd84577b525 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:00.606510 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d91444-186b-4ce0-8a8e-5dd84577b525 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:00.637774 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:00.652291 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e064e96f-30ba-40fd-aecb-4b85db5c5b3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:00.653009 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e064e96f-30ba-40fd-aecb-4b85db5c5b3f None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:00.653291 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e064e96f-30ba-40fd-aecb-4b85db5c5b3f None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:00.653635 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e064e96f-30ba-40fd-aecb-4b85db5c5b3f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:00.654495 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e064e96f-30ba-40fd-aecb-4b85db5c5b3f None admin] RBAC: Adding query filter params (role.id=cd7dd29fcdc844c78544402dc30986fb, scope.project.id=8abb87f2f141451a8c6ed832db82be18, user.id=47b7a9cd615c41d6983a0007b475038c) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:00.654804 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e064e96f-30ba-40fd-aecb-4b85db5c5b3f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:00.656931 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e064e96f-30ba-40fd-aecb-4b85db5c5b3f None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:00.668536 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 333/695] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jun 16 17:31:00 2022] GET /identity/v3/role_assignments?user.id=47b7a9cd615c41d6983a0007b475038c&scope.project.id=8abb87f2f141451a8c6ed832db82be18&role.id=cd7dd29fcdc844c78544402dc30986fb => generated 608 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:00.752002 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100142a1-d100-4d5f-90de-c427c7f40569 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:00.752199 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100142a1-d100-4d5f-90de-c427c7f40569 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:00.752511 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100142a1-d100-4d5f-90de-c427c7f40569 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:00.753930 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 363/696] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:31:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:31:00.759878 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4b8a27-24d3-46ce-bd63-5709f3cd1dea None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:00.760166 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4b8a27-24d3-46ce-bd63-5709f3cd1dea None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:00.760503 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4b8a27-24d3-46ce-bd63-5709f3cd1dea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:00.779479 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-bc4b8a27-24d3-46ce-bd63-5709f3cd1dea None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:31:00.785062 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-bc4b8a27-24d3-46ce-bd63-5709f3cd1dea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:31:00.891783 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 334/697] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:31:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:31:00.918787 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100142a1-d100-4d5f-90de-c427c7f40569 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:01.047994 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100142a1-d100-4d5f-90de-c427c7f40569 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:01.070433 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:01.085717 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c2001c-7107-4a3b-aab0-963e593b2a97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:01.086539 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c2001c-7107-4a3b-aab0-963e593b2a97 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:01.086822 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c2001c-7107-4a3b-aab0-963e593b2a97 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:01.087097 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c2001c-7107-4a3b-aab0-963e593b2a97 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:01.087760 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c2001c-7107-4a3b-aab0-963e593b2a97 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:01.089220 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c2001c-7107-4a3b-aab0-963e593b2a97 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:01.104968 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-54c2001c-7107-4a3b-aab0-963e593b2a97 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 17:31:01.104968 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:01.104968 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 17:31:01.104968 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 17:31:01.104968 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 17:31:01.104968 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 17:31:01.104968 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 17:31:01.104968 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:31:01.104968 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 17:31:01.104968 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:31:01.104968 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:31:01.104968 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:31:01.104968 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:01.104968 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:31:01.104968 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:31:01.104968 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:31:01.106028 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:31:01.106028 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:31:01.106028 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:31:01.106028 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:31:01.106028 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:31:01.106028 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:31:01.106028 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:31:01.106028 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 17:31:01.106028 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 17:31:01.106028 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 17:31:01.106028 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 17:31:01.106028 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:31:01.106028 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:31:01.106028 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:31:01.106028 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:31:01.106028 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:31:01.106864 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:31:01.106864 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:31:01.106864 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:31:01.106864 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:31:01.106864 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:31:01.106864 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:31:01.106864 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:31:01.106864 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:31:01.106864 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:31:01.106864 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:31:01.106864 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:31:01.106864 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 17:31:01.106864 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:31:01.106864 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 17:31:01.106864 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:31:01.106864 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 364/698] 10.222.0.42 () {56 vars in 1129 bytes} [Thu Jun 16 17:31:00 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:31:01.117625 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4b8a27-24d3-46ce-bd63-5709f3cd1dea None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:01.237748 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4b8a27-24d3-46ce-bd63-5709f3cd1dea None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:01.267393 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:01.287165 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d3527b-4053-44fc-b975-38e4807c1b11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:01.287919 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d3527b-4053-44fc-b975-38e4807c1b11 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:01.288212 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d3527b-4053-44fc-b975-38e4807c1b11 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:01.288522 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d3527b-4053-44fc-b975-38e4807c1b11 None admin] PATH_INFO: `/v3/services` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:01.289291 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d3527b-4053-44fc-b975-38e4807c1b11 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:01.289633 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d3527b-4053-44fc-b975-38e4807c1b11 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:01.291127 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d3527b-4053-44fc-b975-38e4807c1b11 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:01.300713 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 335/699] 10.222.0.42 () {56 vars in 1124 bytes} [Thu Jun 16 17:31:01 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:01.303891 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c2001c-7107-4a3b-aab0-963e593b2a97 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:01.354955 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35458f9b-e541-4246-8638-f56712789cbc None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:01.355271 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35458f9b-e541-4246-8638-f56712789cbc None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:01.355512 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35458f9b-e541-4246-8638-f56712789cbc None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:01.357195 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 336/700] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:31:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:31:01.366755 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e0f90e-5c32-4168-9bc2-a0616d411128 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:01.367079 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e0f90e-5c32-4168-9bc2-a0616d411128 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:01.367562 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e0f90e-5c32-4168-9bc2-a0616d411128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:01.398656 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-84e0f90e-5c32-4168-9bc2-a0616d411128 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:31:01.399869 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-84e0f90e-5c32-4168-9bc2-a0616d411128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:31:01.490801 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c2001c-7107-4a3b-aab0-963e593b2a97 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:01.521391 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 337/701] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:31:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:31:01.545137 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e0f90e-5c32-4168-9bc2-a0616d411128 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:01.564995 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fb7f62-ed3e-4ee8-a1e3-77f559168818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:01.565852 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fb7f62-ed3e-4ee8-a1e3-77f559168818 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:01.566134 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fb7f62-ed3e-4ee8-a1e3-77f559168818 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:01.566411 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fb7f62-ed3e-4ee8-a1e3-77f559168818 None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:01.567131 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fb7f62-ed3e-4ee8-a1e3-77f559168818 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:01.567439 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fb7f62-ed3e-4ee8-a1e3-77f559168818 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:01.568894 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fb7f62-ed3e-4ee8-a1e3-77f559168818 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:01.587725 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 365/702] 10.222.0.42 () {56 vars in 1124 bytes} [Thu Jun 16 17:31:01 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:01.715117 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e0f90e-5c32-4168-9bc2-a0616d411128 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:01.740207 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:01.768972 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0282b3e0-de9d-4f65-bf11-4cd87d2a5690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:01.769827 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0282b3e0-de9d-4f65-bf11-4cd87d2a5690 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:01.770107 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0282b3e0-de9d-4f65-bf11-4cd87d2a5690 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:01.770416 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0282b3e0-de9d-4f65-bf11-4cd87d2a5690 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:01.793472 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0282b3e0-de9d-4f65-bf11-4cd87d2a5690 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:31:01.801431 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-0282b3e0-de9d-4f65-bf11-4cd87d2a5690 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:31:01.801431 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:01.801431 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:31:01.801431 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:31:01.801431 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:31:01.801431 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:31:01.801431 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:31:01.801431 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:31:01.801431 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:31:01.801431 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:31:01.801431 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:31:01.801431 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:31:01.801431 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:31:01.801431 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:31:01.801431 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:31:01.801431 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:31:01.802482 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:31:01.802482 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:31:01.802482 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:31:01.802482 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:31:01.802482 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:31:01.802482 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:31:01.802482 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:31:01.802482 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:31:01.802482 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:31:01.802482 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:31:01.802482 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:31:01.802482 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:31:01.802482 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:31:01.802482 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:31:01.802482 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:31:01.802482 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:31:01.803468 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:31:01.803468 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:31:01.803468 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:31:01.803468 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:31:01.803468 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:31:01.803468 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:31:01.803468 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:31:01.803468 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:31:01.803468 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:31:01.803468 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:31:01.803468 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:31:01.803468 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:31:01.803468 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 338/703] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Jun 16 17:31:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:31:01.812731 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fb7f62-ed3e-4ee8-a1e3-77f559168818 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:01.876246 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:01.951930 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01da934d-cb96-4188-b4c4-cf4ba1c3567b None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:01.952338 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01da934d-cb96-4188-b4c4-cf4ba1c3567b None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:01.953403 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01da934d-cb96-4188-b4c4-cf4ba1c3567b None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:01.959062 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 339/704] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:31:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:31:01.965825 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f6e0a3-d4ca-416c-a57a-2e95c6b0af3e None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:01.966112 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f6e0a3-d4ca-416c-a57a-2e95c6b0af3e None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:01.966389 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f6e0a3-d4ca-416c-a57a-2e95c6b0af3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:01.976771 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fb7f62-ed3e-4ee8-a1e3-77f559168818 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:02.012412 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-34f6e0a3-d4ca-416c-a57a-2e95c6b0af3e None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:31:02.017181 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-34f6e0a3-d4ca-416c-a57a-2e95c6b0af3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:31:02.049259 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c4be29-178c-4908-b954-abf4a29f4284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:02.050039 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c4be29-178c-4908-b954-abf4a29f4284 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:02.050317 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c4be29-178c-4908-b954-abf4a29f4284 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:02.050591 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c4be29-178c-4908-b954-abf4a29f4284 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:02.051334 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c4be29-178c-4908-b954-abf4a29f4284 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:02.051639 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c4be29-178c-4908-b954-abf4a29f4284 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:02.053069 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c4be29-178c-4908-b954-abf4a29f4284 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:02.077519 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 366/705] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jun 16 17:31:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:02.083526 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c4be29-178c-4908-b954-abf4a29f4284 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:02.162051 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 340/706] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:31:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:31:02.201891 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f6e0a3-d4ca-416c-a57a-2e95c6b0af3e None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:02.264728 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c4be29-178c-4908-b954-abf4a29f4284 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:02.288797 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:02.321703 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f6e0a3-d4ca-416c-a57a-2e95c6b0af3e None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:02.332968 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:02.353925 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf947da8-00b1-42c6-b626-873d210b0042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:02.354742 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf947da8-00b1-42c6-b626-873d210b0042 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:02.355020 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf947da8-00b1-42c6-b626-873d210b0042 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:02.355295 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf947da8-00b1-42c6-b626-873d210b0042 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:02.369157 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43671878-7646-4c30-a75d-fb83136e01f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:02.369969 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43671878-7646-4c30-a75d-fb83136e01f2 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:02.370248 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43671878-7646-4c30-a75d-fb83136e01f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:02.370518 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43671878-7646-4c30-a75d-fb83136e01f2 None admin] PATH_INFO: `/v3/services/network` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:02.371229 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43671878-7646-4c30-a75d-fb83136e01f2 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:02.372639 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43671878-7646-4c30-a75d-fb83136e01f2 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:02.382413 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf947da8-00b1-42c6-b626-873d210b0042 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:02.383856 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf947da8-00b1-42c6-b626-873d210b0042 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:02.395620 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-43671878-7646-4c30-a75d-fb83136e01f2 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 17:31:02.395620 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:02.395620 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 17:31:02.395620 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 17:31:02.395620 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 17:31:02.395620 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 17:31:02.395620 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 17:31:02.395620 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:31:02.395620 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 17:31:02.395620 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:31:02.395620 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:31:02.395620 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:31:02.395620 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:02.395620 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:31:02.395620 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:31:02.395620 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:31:02.395620 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:31:02.397126 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:31:02.397126 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:31:02.397126 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:31:02.397126 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:31:02.397126 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:31:02.397126 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:31:02.397126 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 17:31:02.397126 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 17:31:02.397126 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 17:31:02.397126 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 17:31:02.397126 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:31:02.397126 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:31:02.397126 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:31:02.397126 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:31:02.397126 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:31:02.397126 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:31:02.397972 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:31:02.397972 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:31:02.397972 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:31:02.397972 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:31:02.397972 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:31:02.397972 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:31:02.397972 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:31:02.397972 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:31:02.397972 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:31:02.397972 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:31:02.397972 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 17:31:02.397972 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:31:02.397972 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 17:31:02.397972 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:31:02.415277 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-cf947da8-00b1-42c6-b626-873d210b0042 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:31:02.415277 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:02.415277 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:31:02.415277 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:31:02.415277 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:31:02.415277 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:31:02.415277 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:31:02.415277 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:31:02.415277 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:31:02.415277 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:31:02.415277 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:31:02.415277 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:31:02.415277 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:31:02.415277 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:31:02.415277 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:31:02.417373 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:31:02.417373 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:31:02.417373 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:31:02.417373 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:31:02.417373 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:31:02.417373 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:31:02.417373 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:31:02.417373 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:31:02.417373 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:31:02.417373 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:31:02.417373 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:31:02.417373 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:31:02.417373 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:31:02.417373 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:31:02.417373 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:31:02.417373 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:31:02.417373 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:31:02.419520 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:31:02.419520 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:31:02.419520 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:31:02.419520 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:31:02.420029 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 341/707] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jun 16 17:31:02 2022] GET /identity/v3/services/network => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:31:02.420172 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 367/708] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jun 16 17:31:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:31:02.422566 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43671878-7646-4c30-a75d-fb83136e01f2 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:02.434000 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf947da8-00b1-42c6-b626-873d210b0042 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:02.554767 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43671878-7646-4c30-a75d-fb83136e01f2 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:02.587484 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf947da8-00b1-42c6-b626-873d210b0042 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:02.605734 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7469032-0db0-4979-8ba3-ed8396e5fb47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:02.606422 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7469032-0db0-4979-8ba3-ed8396e5fb47 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:02.606705 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7469032-0db0-4979-8ba3-ed8396e5fb47 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:02.606980 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7469032-0db0-4979-8ba3-ed8396e5fb47 None admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:02.607705 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7469032-0db0-4979-8ba3-ed8396e5fb47 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:02.608009 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7469032-0db0-4979-8ba3-ed8396e5fb47 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:02.609559 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7469032-0db0-4979-8ba3-ed8396e5fb47 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:02.623911 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812e91c9-1ce8-499b-be2f-4cab7d9cb76b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:02.624650 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812e91c9-1ce8-499b-be2f-4cab7d9cb76b None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:02.624940 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812e91c9-1ce8-499b-be2f-4cab7d9cb76b None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:02.625218 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812e91c9-1ce8-499b-be2f-4cab7d9cb76b None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:02.626141 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812e91c9-1ce8-499b-be2f-4cab7d9cb76b None admin] RBAC: Adding query filter params (name=network) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:02.626453 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812e91c9-1ce8-499b-be2f-4cab7d9cb76b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:02.627842 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812e91c9-1ce8-499b-be2f-4cab7d9cb76b None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:02.640902 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 342/709] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jun 16 17:31:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:02.641343 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 368/710] 10.222.0.42 () {56 vars in 1112 bytes} [Thu Jun 16 17:31:02 2022] GET /identity/v3/services?name=network => generated 124 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:02.643243 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7469032-0db0-4979-8ba3-ed8396e5fb47 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:02.643790 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812e91c9-1ce8-499b-be2f-4cab7d9cb76b None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:02.673681 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:02.682029 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:02.770072 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812e91c9-1ce8-499b-be2f-4cab7d9cb76b None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:02.801542 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7469032-0db0-4979-8ba3-ed8396e5fb47 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:02.805868 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32100d54-5813-449d-afa1-4df807cc3420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:02.806754 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32100d54-5813-449d-afa1-4df807cc3420 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:02.807202 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32100d54-5813-449d-afa1-4df807cc3420 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:02.807652 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32100d54-5813-449d-afa1-4df807cc3420 None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:02.808533 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32100d54-5813-449d-afa1-4df807cc3420 None admin] RBAC: Adding query filter params (type=network) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:02.808991 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32100d54-5813-449d-afa1-4df807cc3420 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:02.810500 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32100d54-5813-449d-afa1-4df807cc3420 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:02.815799 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 369/711] 10.222.0.42 () {56 vars in 1112 bytes} [Thu Jun 16 17:31:02 2022] GET /identity/v3/services?type=network => generated 124 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:02.824415 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761586fb-4a41-4c97-8fb3-7c68a7e8fcd0 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:02.825279 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761586fb-4a41-4c97-8fb3-7c68a7e8fcd0 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:02.825772 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761586fb-4a41-4c97-8fb3-7c68a7e8fcd0 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:02.827799 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 370/712] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:31:02 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:31:02.833383 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e1d985-68ff-4c6a-8541-9a4145caea29 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:02.834035 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e1d985-68ff-4c6a-8541-9a4145caea29 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:02.834520 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e1d985-68ff-4c6a-8541-9a4145caea29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:02.855795 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1568cd-c163-431f-9d58-97005cd5cac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:02.857084 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1568cd-c163-431f-9d58-97005cd5cac1 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:02.863758 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1568cd-c163-431f-9d58-97005cd5cac1 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:02.864417 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1568cd-c163-431f-9d58-97005cd5cac1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:02.872461 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-28e1d985-68ff-4c6a-8541-9a4145caea29 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:31:02.879474 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-28e1d985-68ff-4c6a-8541-9a4145caea29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:31:02.881269 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1568cd-c163-431f-9d58-97005cd5cac1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:02.883043 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1568cd-c163-431f-9d58-97005cd5cac1 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:02.911687 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-3f1568cd-c163-431f-9d58-97005cd5cac1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 17:31:02.911687 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:02.911687 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:31:02.911687 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:31:02.911687 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:31:02.911687 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:31:02.911687 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:31:02.911687 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:31:02.911687 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:31:02.911687 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:31:02.911687 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:31:02.911687 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:31:02.911687 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:31:02.911687 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:31:02.911687 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:31:02.911687 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:31:02.912795 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:31:02.912795 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:31:02.912795 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:31:02.912795 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:31:02.912795 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:31:02.912795 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:31:02.912795 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:31:02.912795 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:31:02.912795 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:31:02.912795 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:31:02.912795 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:31:02.912795 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:31:02.912795 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:31:02.912795 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:31:02.912795 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:31:02.912795 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:31:02.912795 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:31:02.912795 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:31:02.913797 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:31:02.913797 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:31:02.913797 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:31:02.913797 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:31:02.913797 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:31:02.913797 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:31:02.913797 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:31:02.913797 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:31:02.913797 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 17:31:02.913797 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:31:02.913797 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 343/713] 10.222.0.42 () {56 vars in 1093 bytes} [Thu Jun 16 17:31:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:31:02.929557 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1568cd-c163-431f-9d58-97005cd5cac1 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:03.016801 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 371/714] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:31:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 17:31:03.052752 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e1d985-68ff-4c6a-8541-9a4145caea29 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:03.069399 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:03.083022 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1568cd-c163-431f-9d58-97005cd5cac1 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:03.089546 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:03.143537 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba3e6e6-e898-42d8-b212-3f04b5dee04b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:03.144542 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba3e6e6-e898-42d8-b212-3f04b5dee04b None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:03.145040 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba3e6e6-e898-42d8-b212-3f04b5dee04b None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:03.145457 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba3e6e6-e898-42d8-b212-3f04b5dee04b None admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:03.146489 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba3e6e6-e898-42d8-b212-3f04b5dee04b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:03.147122 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba3e6e6-e898-42d8-b212-3f04b5dee04b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:03.150109 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba3e6e6-e898-42d8-b212-3f04b5dee04b None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:03.187959 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 344/715] 10.222.0.42 () {56 vars in 1133 bytes} [Thu Jun 16 17:31:02 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:03.201546 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba3e6e6-e898-42d8-b212-3f04b5dee04b None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:03.240243 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e1d985-68ff-4c6a-8541-9a4145caea29 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:03.315786 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba3e6e6-e898-42d8-b212-3f04b5dee04b None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:03.322473 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a6fc3f-1d31-4288-94b1-8bbbd6101bbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:03.323361 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a6fc3f-1d31-4288-94b1-8bbbd6101bbb None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:03.323717 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a6fc3f-1d31-4288-94b1-8bbbd6101bbb None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:03.324065 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a6fc3f-1d31-4288-94b1-8bbbd6101bbb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:03.353295 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a6fc3f-1d31-4288-94b1-8bbbd6101bbb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:31:03.356634 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f3ede4-6e19-4785-bda0-d57459ecfff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:03.357523 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f3ede4-6e19-4785-bda0-d57459ecfff3 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:03.357983 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f3ede4-6e19-4785-bda0-d57459ecfff3 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:03.358428 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f3ede4-6e19-4785-bda0-d57459ecfff3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:03.361614 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-71a6fc3f-1d31-4288-94b1-8bbbd6101bbb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:31:03.361614 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:03.361614 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:31:03.361614 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:31:03.361614 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:31:03.361614 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:31:03.361614 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:31:03.361614 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:31:03.361614 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:31:03.361614 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:31:03.361614 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:31:03.361614 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:31:03.361614 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:31:03.361614 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:31:03.361614 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:31:03.361614 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:31:03.362543 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:31:03.362543 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:31:03.362543 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:31:03.362543 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:31:03.362543 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:31:03.362543 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:31:03.362543 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:31:03.362543 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:31:03.362543 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:31:03.362543 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:31:03.362543 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:31:03.362543 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:31:03.362543 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:31:03.362543 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:31:03.362543 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:31:03.362543 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:31:03.363497 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:31:03.363497 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:31:03.363497 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:31:03.363497 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:31:03.363497 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:31:03.363497 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:31:03.363497 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:31:03.363497 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:31:03.363497 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:31:03.363497 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:31:03.363497 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:31:03.363497 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:31:03.372840 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 372/716] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Jun 16 17:31:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:31:03.379047 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a6fc3f-1d31-4288-94b1-8bbbd6101bbb None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:03.384520 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:03.442587 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f3ede4-6e19-4785-bda0-d57459ecfff3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:03.444591 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f3ede4-6e19-4785-bda0-d57459ecfff3 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:03.556476 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 345/717] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jun 16 17:31:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:31:03.561598 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f3ede4-6e19-4785-bda0-d57459ecfff3 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:03.595295 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:03.644769 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:03.674402 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a6fc3f-1d31-4288-94b1-8bbbd6101bbb None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:03.682845 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f3ede4-6e19-4785-bda0-d57459ecfff3 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:03.717359 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76176c97-7956-4750-a31b-d1653c84fed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:03.718000 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76176c97-7956-4750-a31b-d1653c84fed2 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:03.718232 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76176c97-7956-4750-a31b-d1653c84fed2 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:03.718545 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76176c97-7956-4750-a31b-d1653c84fed2 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:03.719385 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76176c97-7956-4750-a31b-d1653c84fed2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:03.719701 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76176c97-7956-4750-a31b-d1653c84fed2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:03.721351 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76176c97-7956-4750-a31b-d1653c84fed2 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:03.730277 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcbc866-b1d8-4861-a825-10f448dca776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:03.730849 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 373/718] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Jun 16 17:31:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:31:03.731760 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcbc866-b1d8-4861-a825-10f448dca776 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:03.732247 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcbc866-b1d8-4861-a825-10f448dca776 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:03.732364 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ed4f0e-5ff0-4014-bd66-c4f338e758e0 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:03.732604 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ed4f0e-5ff0-4014-bd66-c4f338e758e0 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:03.732885 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ed4f0e-5ff0-4014-bd66-c4f338e758e0 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:03.733312 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcbc866-b1d8-4861-a825-10f448dca776 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:03.734251 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 374/719] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:31:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:31:03.739191 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ed4f0e-5ff0-4014-bd66-c4f338e758e0 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:03.746862 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcbc866-b1d8-4861-a825-10f448dca776 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:03.749024 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcbc866-b1d8-4861-a825-10f448dca776 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:03.774118 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-ebcbc866-b1d8-4861-a825-10f448dca776 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:31:03.774118 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:03.774118 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:31:03.774118 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:31:03.774118 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:31:03.774118 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:31:03.774118 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:31:03.774118 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:31:03.774118 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:31:03.774118 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:31:03.774118 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:31:03.774118 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:31:03.774118 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:31:03.774118 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:31:03.774118 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:31:03.775107 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:31:03.775107 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:31:03.775107 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:31:03.775107 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:31:03.775107 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:31:03.775107 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:31:03.775107 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:31:03.775107 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:31:03.775107 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:31:03.775107 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:31:03.775107 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:31:03.775107 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:31:03.775107 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:31:03.775107 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:31:03.775107 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:31:03.775107 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:31:03.775107 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:31:03.776032 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:31:03.776032 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:31:03.776032 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:31:03.776032 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:31:03.782286 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 346/720] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jun 16 17:31:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:31:03.782286 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d530bac-2b02-4d2d-9555-fc07dfae1083 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:03.782286 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d530bac-2b02-4d2d-9555-fc07dfae1083 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:03.782286 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d530bac-2b02-4d2d-9555-fc07dfae1083 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:03.807150 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-2d530bac-2b02-4d2d-9555-fc07dfae1083 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:31:03.809469 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-2d530bac-2b02-4d2d-9555-fc07dfae1083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:31:03.867810 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ed4f0e-5ff0-4014-bd66-c4f338e758e0 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:03.896699 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 347/721] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:31:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:31:03.897681 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d530bac-2b02-4d2d-9555-fc07dfae1083 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:03.912056 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0214a8-a6de-41ad-ae04-e194904f6b9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:03.913356 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0214a8-a6de-41ad-ae04-e194904f6b9a None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:03.913861 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0214a8-a6de-41ad-ae04-e194904f6b9a None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:03.914369 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0214a8-a6de-41ad-ae04-e194904f6b9a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:03.929130 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0214a8-a6de-41ad-ae04-e194904f6b9a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:03.930956 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0214a8-a6de-41ad-ae04-e194904f6b9a None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:03.933682 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:03.949226 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:03.958585 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-5c0214a8-a6de-41ad-ae04-e194904f6b9a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:31:03.958585 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:03.958585 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:31:03.958585 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:31:03.958585 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:31:03.958585 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:31:03.958585 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:31:03.958585 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:31:03.958585 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:31:03.958585 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:31:03.958585 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:31:03.958585 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:31:03.958585 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:31:03.958585 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:31:03.958585 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:31:03.959643 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:31:03.959643 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:31:03.959643 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:31:03.959643 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:31:03.959643 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:31:03.959643 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:31:03.959643 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:31:03.959643 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:31:03.959643 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:31:03.959643 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:31:03.959643 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:31:03.959643 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:31:03.959643 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:31:03.959643 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:31:03.959643 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:31:03.959643 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:31:03.959643 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:31:03.960614 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:31:03.960614 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:31:03.960614 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:31:03.960614 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:31:03.962070 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 375/722] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jun 16 17:31:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:31:03.963693 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0214a8-a6de-41ad-ae04-e194904f6b9a None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:04.057294 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d530bac-2b02-4d2d-9555-fc07dfae1083 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:04.094671 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0214a8-a6de-41ad-ae04-e194904f6b9a None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:04.102288 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f48416-8fad-4379-a785-8614fa03da56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:04.102966 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f48416-8fad-4379-a785-8614fa03da56 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:04.103250 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f48416-8fad-4379-a785-8614fa03da56 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:04.103527 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f48416-8fad-4379-a785-8614fa03da56 None admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:04.104259 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f48416-8fad-4379-a785-8614fa03da56 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:04.104610 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f48416-8fad-4379-a785-8614fa03da56 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:04.106007 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f48416-8fad-4379-a785-8614fa03da56 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:04.123787 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 348/723] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jun 16 17:31:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:04.124768 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f48416-8fad-4379-a785-8614fa03da56 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:04.139573 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ea0633-6f89-44f2-9415-d31b4d2e61cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:04.141712 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ea0633-6f89-44f2-9415-d31b4d2e61cc None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:04.142026 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ea0633-6f89-44f2-9415-d31b4d2e61cc None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:04.142301 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ea0633-6f89-44f2-9415-d31b4d2e61cc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:04.153361 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ea0633-6f89-44f2-9415-d31b4d2e61cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:31:04.161908 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-63ea0633-6f89-44f2-9415-d31b4d2e61cc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:31:04.161908 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:04.161908 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:31:04.161908 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:31:04.161908 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:31:04.161908 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:31:04.161908 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:31:04.161908 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:31:04.161908 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:31:04.161908 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:31:04.161908 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:31:04.161908 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:31:04.161908 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:31:04.161908 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:31:04.161908 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:31:04.161908 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:31:04.162854 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:31:04.162854 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:31:04.162854 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:31:04.162854 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:31:04.162854 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:31:04.162854 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:31:04.162854 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:31:04.162854 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:31:04.162854 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:31:04.162854 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:31:04.162854 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:31:04.162854 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:31:04.162854 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:31:04.162854 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:31:04.162854 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:31:04.162854 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:31:04.163706 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:31:04.163706 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:31:04.163706 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:31:04.163706 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:31:04.163706 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:31:04.163706 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:31:04.163706 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:31:04.163706 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:31:04.163706 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:31:04.163706 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:31:04.163706 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:31:04.163706 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:31:04.163706 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 376/724] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Jun 16 17:31:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:31:04.164534 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ea0633-6f89-44f2-9415-d31b4d2e61cc None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:04.223360 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:04.244793 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:04.261430 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f48416-8fad-4379-a785-8614fa03da56 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:04.292933 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ea0633-6f89-44f2-9415-d31b4d2e61cc None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:04.317403 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cf57cc-815a-4070-85c3-19d9f172b59f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:04.318440 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cf57cc-815a-4070-85c3-19d9f172b59f None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:04.319177 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cf57cc-815a-4070-85c3-19d9f172b59f None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:04.319360 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cf57cc-815a-4070-85c3-19d9f172b59f None admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:04.320632 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cf57cc-815a-4070-85c3-19d9f172b59f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:04.321209 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cf57cc-815a-4070-85c3-19d9f172b59f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:04.323424 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cf57cc-815a-4070-85c3-19d9f172b59f None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:04.348030 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 349/725] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jun 16 17:31:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:04.348030 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395b8405-cebd-4d3a-9cab-a1159207a277 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:04.348030 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395b8405-cebd-4d3a-9cab-a1159207a277 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:04.348030 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395b8405-cebd-4d3a-9cab-a1159207a277 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:04.348030 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 350/726] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:31:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:31:04.348517 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395b8405-cebd-4d3a-9cab-a1159207a277 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:04.356077 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d35f1f-ddc5-410f-90ee-3c12d826bcec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:04.356946 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d35f1f-ddc5-410f-90ee-3c12d826bcec None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:04.357225 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d35f1f-ddc5-410f-90ee-3c12d826bcec None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:04.357838 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d35f1f-ddc5-410f-90ee-3c12d826bcec None admin] PATH_INFO: `/v3/projects/service` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:04.366288 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d35f1f-ddc5-410f-90ee-3c12d826bcec None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:04.367739 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d35f1f-ddc5-410f-90ee-3c12d826bcec None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:04.382799 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-54d35f1f-ddc5-410f-90ee-3c12d826bcec None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:31:04.382799 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:04.382799 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:31:04.382799 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:31:04.382799 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:31:04.382799 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:31:04.382799 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:31:04.382799 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:31:04.382799 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:31:04.382799 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:31:04.382799 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:31:04.382799 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:31:04.382799 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:31:04.382799 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:31:04.382799 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:31:04.383924 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:31:04.383924 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:31:04.383924 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:31:04.383924 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:31:04.383924 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:31:04.383924 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:31:04.383924 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:31:04.383924 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:31:04.383924 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:31:04.383924 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:31:04.383924 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:31:04.383924 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:31:04.383924 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:31:04.383924 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:31:04.383924 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:31:04.383924 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:31:04.383924 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:31:04.385119 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:31:04.385119 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:31:04.385119 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:31:04.385119 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:31:04.385119 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:31:04.385119 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:31:04.385119 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:31:04.385119 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:31:04.385119 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 377/727] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jun 16 17:31:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:31:04.385119 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d35f1f-ddc5-410f-90ee-3c12d826bcec None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:04.465028 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395b8405-cebd-4d3a-9cab-a1159207a277 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:04.496215 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d35f1f-ddc5-410f-90ee-3c12d826bcec None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:04.503467 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38c78c4-d46c-4fa0-a1bb-c94299d49159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:04.504345 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38c78c4-d46c-4fa0-a1bb-c94299d49159 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:04.504658 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38c78c4-d46c-4fa0-a1bb-c94299d49159 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:04.504935 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38c78c4-d46c-4fa0-a1bb-c94299d49159 None admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:04.505681 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38c78c4-d46c-4fa0-a1bb-c94299d49159 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:04.505984 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38c78c4-d46c-4fa0-a1bb-c94299d49159 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:04.507347 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38c78c4-d46c-4fa0-a1bb-c94299d49159 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:04.511849 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:04.516735 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 351/728] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jun 16 17:31:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:04.522355 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149b611b-25f8-4014-a3aa-bf7a685fb064 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:04.522684 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149b611b-25f8-4014-a3aa-bf7a685fb064 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:04.522924 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149b611b-25f8-4014-a3aa-bf7a685fb064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:04.537251 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b3095e-4ac5-4574-bff2-64db09690023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:04.538065 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b3095e-4ac5-4574-bff2-64db09690023 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:04.538348 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b3095e-4ac5-4574-bff2-64db09690023 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:04.538622 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b3095e-4ac5-4574-bff2-64db09690023 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:04.539926 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-149b611b-25f8-4014-a3aa-bf7a685fb064 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:31:04.541153 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-149b611b-25f8-4014-a3aa-bf7a685fb064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:31:04.546255 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b3095e-4ac5-4574-bff2-64db09690023 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:04.548403 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b3095e-4ac5-4574-bff2-64db09690023 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:04.552170 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:04.564426 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-e1b3095e-4ac5-4574-bff2-64db09690023 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 17:31:04.564426 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:04.564426 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:31:04.564426 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:31:04.564426 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:31:04.564426 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:31:04.564426 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:31:04.564426 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:31:04.564426 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:31:04.564426 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:31:04.564426 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:31:04.564426 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:31:04.564426 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:31:04.564426 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:31:04.564426 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:31:04.564426 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:31:04.565429 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:31:04.565429 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:31:04.565429 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:31:04.565429 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:31:04.565429 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:31:04.565429 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:31:04.565429 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:31:04.565429 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:31:04.565429 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:31:04.565429 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:31:04.565429 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:31:04.565429 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:31:04.565429 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:31:04.565429 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:31:04.565429 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:31:04.565429 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:31:04.565429 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:31:04.565429 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:31:04.566388 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:31:04.566388 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:31:04.566388 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:31:04.566388 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:31:04.566388 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:31:04.566388 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:31:04.566388 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:31:04.566388 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:31:04.566388 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 17:31:04.566388 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:31:04.570592 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 378/729] 10.222.0.42 () {56 vars in 1117 bytes} [Thu Jun 16 17:31:04 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:31:04.571052 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b3095e-4ac5-4574-bff2-64db09690023 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:04.617481 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 352/730] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:31:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:31:04.618340 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149b611b-25f8-4014-a3aa-bf7a685fb064 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:04.696787 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b3095e-4ac5-4574-bff2-64db09690023 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:04.737095 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149b611b-25f8-4014-a3aa-bf7a685fb064 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:04.743502 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99564cda-5509-4244-9e35-88111020d9ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:04.744184 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99564cda-5509-4244-9e35-88111020d9ca None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:04.744500 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99564cda-5509-4244-9e35-88111020d9ca None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:04.744792 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99564cda-5509-4244-9e35-88111020d9ca None admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:04.745583 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99564cda-5509-4244-9e35-88111020d9ca None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:04.745896 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99564cda-5509-4244-9e35-88111020d9ca None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:04.747349 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99564cda-5509-4244-9e35-88111020d9ca None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:04.764235 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 379/731] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Jun 16 17:31:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:04.764235 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99564cda-5509-4244-9e35-88111020d9ca None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:04.780428 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bf4a77-af9e-4d08-89e6-6b557e1dc8cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:04.781957 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bf4a77-af9e-4d08-89e6-6b557e1dc8cc None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:04.782465 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bf4a77-af9e-4d08-89e6-6b557e1dc8cc None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:04.782575 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bf4a77-af9e-4d08-89e6-6b557e1dc8cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:04.794222 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bf4a77-af9e-4d08-89e6-6b557e1dc8cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:04.795795 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bf4a77-af9e-4d08-89e6-6b557e1dc8cc None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:04.799244 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:04.801617 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:04.819376 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-f2bf4a77-af9e-4d08-89e6-6b557e1dc8cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:31:04.819376 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:04.819376 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:31:04.819376 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:31:04.819376 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:31:04.819376 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:31:04.819376 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:31:04.819376 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:31:04.819376 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:31:04.819376 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:31:04.819376 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:31:04.819376 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:31:04.819376 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:31:04.819376 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:31:04.819376 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:31:04.820336 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:31:04.820336 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:31:04.820336 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:31:04.820336 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:31:04.820336 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:31:04.820336 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:31:04.820336 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:31:04.820336 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:31:04.820336 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:31:04.820336 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:31:04.820336 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:31:04.820336 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:31:04.820336 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:31:04.820336 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:31:04.820336 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:31:04.820336 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:31:04.820336 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:31:04.826443 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:31:04.826443 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:31:04.826443 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:31:04.826443 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:31:04.826443 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 353/732] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jun 16 17:31:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:31:04.826443 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bf4a77-af9e-4d08-89e6-6b557e1dc8cc None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:04.880874 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99564cda-5509-4244-9e35-88111020d9ca None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:04.929610 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811fb61e-a534-4748-ad8b-cf0a7df310da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:04.930291 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811fb61e-a534-4748-ad8b-cf0a7df310da None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:04.930574 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811fb61e-a534-4748-ad8b-cf0a7df310da None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:04.930853 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811fb61e-a534-4748-ad8b-cf0a7df310da None admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:04.931691 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811fb61e-a534-4748-ad8b-cf0a7df310da None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:04.932002 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811fb61e-a534-4748-ad8b-cf0a7df310da None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:04.933477 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811fb61e-a534-4748-ad8b-cf0a7df310da None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:04.957084 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 380/733] 10.222.0.42 () {56 vars in 1149 bytes} [Thu Jun 16 17:31:04 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:04.957312 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bf4a77-af9e-4d08-89e6-6b557e1dc8cc None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:04.957413 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811fb61e-a534-4748-ad8b-cf0a7df310da None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:04.993938 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8cd1a5-6fcb-4afc-8f58-8628fba69d5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:04.994617 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8cd1a5-6fcb-4afc-8f58-8628fba69d5f None admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:04.994903 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8cd1a5-6fcb-4afc-8f58-8628fba69d5f None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:04.995180 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8cd1a5-6fcb-4afc-8f58-8628fba69d5f None admin] PATH_INFO: `/v3/services` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:04.995959 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8cd1a5-6fcb-4afc-8f58-8628fba69d5f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:04.997458 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8cd1a5-6fcb-4afc-8f58-8628fba69d5f None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:05.024400 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 354/734] 10.222.0.42 () {60 vars in 1141 bytes} [Thu Jun 16 17:31:04 2022] POST /identity/v3/services => generated 251 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:31:05.024400 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8cd1a5-6fcb-4afc-8f58-8628fba69d5f None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:05.105170 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:05.106399 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:05.117179 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811fb61e-a534-4748-ad8b-cf0a7df310da None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:05.177908 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8cd1a5-6fcb-4afc-8f58-8628fba69d5f None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:05.181845 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a390cca-82d5-4785-a32b-c4a4b237e4c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:05.182784 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a390cca-82d5-4785-a32b-c4a4b237e4c7 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:05.183168 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a390cca-82d5-4785-a32b-c4a4b237e4c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:05.183546 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a390cca-82d5-4785-a32b-c4a4b237e4c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:05.184554 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a390cca-82d5-4785-a32b-c4a4b237e4c7 None admin] RBAC: Adding query filter params (role.id=3ca2dcbbd29d4a31a3ab78f2299bc239, scope.project.id=8abb87f2f141451a8c6ed832db82be18, user.id=f86c16179a70474a817be9fda33ece9d) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:05.184944 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a390cca-82d5-4785-a32b-c4a4b237e4c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:05.186759 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a390cca-82d5-4785-a32b-c4a4b237e4c7 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:05.195731 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 381/735] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jun 16 17:31:04 2022] GET /identity/v3/role_assignments?user.id=f86c16179a70474a817be9fda33ece9d&scope.project.id=8abb87f2f141451a8c6ed832db82be18&role.id=3ca2dcbbd29d4a31a3ab78f2299bc239 => generated 259 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:05.196776 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a390cca-82d5-4785-a32b-c4a4b237e4c7 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:05.245154 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1dd67c-b70d-4d08-8b37-dc6ef07f6a92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:05.245154 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1dd67c-b70d-4d08-8b37-dc6ef07f6a92 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:05.245154 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1dd67c-b70d-4d08-8b37-dc6ef07f6a92 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:05.245154 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1dd67c-b70d-4d08-8b37-dc6ef07f6a92 None admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:05.246796 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1dd67c-b70d-4d08-8b37-dc6ef07f6a92 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:05.247162 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1dd67c-b70d-4d08-8b37-dc6ef07f6a92 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:05.248747 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1dd67c-b70d-4d08-8b37-dc6ef07f6a92 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:05.264694 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 355/736] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jun 16 17:31:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:05.273554 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1dd67c-b70d-4d08-8b37-dc6ef07f6a92 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:05.346218 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a390cca-82d5-4785-a32b-c4a4b237e4c7 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:05.404063 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73f73f7-5f96-4b92-a0b6-f401ec2f3775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:05.405343 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73f73f7-5f96-4b92-a0b6-f401ec2f3775 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:05.405693 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73f73f7-5f96-4b92-a0b6-f401ec2f3775 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:05.406253 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73f73f7-5f96-4b92-a0b6-f401ec2f3775 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:05.429991 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1dd67c-b70d-4d08-8b37-dc6ef07f6a92 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:05.434955 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:05.460978 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:05.477293 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f37386f-c59e-4715-96c6-6d23ebd3c59d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:05.478133 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f37386f-c59e-4715-96c6-6d23ebd3c59d None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:05.478445 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f37386f-c59e-4715-96c6-6d23ebd3c59d None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:05.478723 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f37386f-c59e-4715-96c6-6d23ebd3c59d None admin] PATH_INFO: `/v3/users/swift` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:05.487956 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f37386f-c59e-4715-96c6-6d23ebd3c59d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:05.489750 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f37386f-c59e-4715-96c6-6d23ebd3c59d None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:05.500270 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73f73f7-5f96-4b92-a0b6-f401ec2f3775 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:05.502007 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73f73f7-5f96-4b92-a0b6-f401ec2f3775 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:05.507644 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-0f37386f-c59e-4715-96c6-6d23ebd3c59d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 16 17:31:05.507644 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:05.507644 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:31:05.507644 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:31:05.507644 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:31:05.507644 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:31:05.507644 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:31:05.507644 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:31:05.507644 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:31:05.507644 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:31:05.507644 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:31:05.507644 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:31:05.507644 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:31:05.507644 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:31:05.507644 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:31:05.507644 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:31:05.508838 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:31:05.508838 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:31:05.508838 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:31:05.508838 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:31:05.508838 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:31:05.508838 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:31:05.508838 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:31:05.508838 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:31:05.508838 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:31:05.508838 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:31:05.508838 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:31:05.508838 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:31:05.508838 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:31:05.508838 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:31:05.508838 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:31:05.508838 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:31:05.508838 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:31:05.508838 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:31:05.509913 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:31:05.509913 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:31:05.509913 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:31:05.509913 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:31:05.509913 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:31:05.509913 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:31:05.509913 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:31:05.509913 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:31:05.509913 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 16 17:31:05.509913 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:31:05.511169 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 356/737] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Jun 16 17:31:05 2022] GET /identity/v3/users/swift => generated 83 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:31:05.512475 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c01a41-30bd-469e-8190-80df6b0f4e6c None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:05.512760 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c01a41-30bd-469e-8190-80df6b0f4e6c None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:05.513033 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c01a41-30bd-469e-8190-80df6b0f4e6c None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:05.519181 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 357/738] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:31:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:31:05.519181 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c01a41-30bd-469e-8190-80df6b0f4e6c None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:05.619741 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 382/739] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jun 16 17:31:05 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:31:05.621099 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1d75d4-907b-47a6-a8f4-8b97e59171e5 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:05.621390 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1d75d4-907b-47a6-a8f4-8b97e59171e5 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:05.621673 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1d75d4-907b-47a6-a8f4-8b97e59171e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:05.637291 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c01a41-30bd-469e-8190-80df6b0f4e6c None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:05.638924 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-ae1d75d4-907b-47a6-a8f4-8b97e59171e5 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:31:05.641544 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-ae1d75d4-907b-47a6-a8f4-8b97e59171e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:31:05.677521 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f24da7-6df6-44a5-aaf7-fe734bf114b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:05.678649 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f24da7-6df6-44a5-aaf7-fe734bf114b0 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:05.679169 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f24da7-6df6-44a5-aaf7-fe734bf114b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:05.679663 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f24da7-6df6-44a5-aaf7-fe734bf114b0 None admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:05.680776 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f24da7-6df6-44a5-aaf7-fe734bf114b0 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:05.681283 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f24da7-6df6-44a5-aaf7-fe734bf114b0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:05.682969 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f24da7-6df6-44a5-aaf7-fe734bf114b0 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:05.692153 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:05.706074 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 358/740] 10.222.0.42 () {56 vars in 1135 bytes} [Thu Jun 16 17:31:05 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:31:05.707472 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f24da7-6df6-44a5-aaf7-fe734bf114b0 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:05.727946 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 383/741] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:31:05 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:31:05.727946 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1d75d4-907b-47a6-a8f4-8b97e59171e5 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:05.759478 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:05.841563 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f24da7-6df6-44a5-aaf7-fe734bf114b0 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:05.867584 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1d75d4-907b-47a6-a8f4-8b97e59171e5 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:05.891654 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96057abd-c767-4ff0-9def-feed77b7cbdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:05.892534 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96057abd-c767-4ff0-9def-feed77b7cbdd None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:05.892919 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96057abd-c767-4ff0-9def-feed77b7cbdd None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:05.893258 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96057abd-c767-4ff0-9def-feed77b7cbdd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:05.905192 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96057abd-c767-4ff0-9def-feed77b7cbdd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:05.907621 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96057abd-c767-4ff0-9def-feed77b7cbdd None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:05.912816 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baee04a4-a856-4eb9-a936-e8301d22fffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:05.913708 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baee04a4-a856-4eb9-a936-e8301d22fffe None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:05.914196 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baee04a4-a856-4eb9-a936-e8301d22fffe None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:05.915186 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baee04a4-a856-4eb9-a936-e8301d22fffe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:05.930116 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-96057abd-c767-4ff0-9def-feed77b7cbdd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:31:05.930116 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:05.930116 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:31:05.930116 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:31:05.930116 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:31:05.930116 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:31:05.930116 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:31:05.930116 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:31:05.930116 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:31:05.930116 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:31:05.930116 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:31:05.930116 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:31:05.930116 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:31:05.930116 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:31:05.930116 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:31:05.931033 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:31:05.931033 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:31:05.931033 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:31:05.931033 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:31:05.931033 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:31:05.931033 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:31:05.931033 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:31:05.931033 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:31:05.931033 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:31:05.931033 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:31:05.931033 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:31:05.931033 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:31:05.931033 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:31:05.931033 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:31:05.931033 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:31:05.931033 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:31:05.931033 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:31:05.931934 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:31:05.931934 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:31:05.931934 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:31:05.931934 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:31:05.931934 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 359/742] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jun 16 17:31:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:31:05.931934 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96057abd-c767-4ff0-9def-feed77b7cbdd None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:05.976539 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:05.999294 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baee04a4-a856-4eb9-a936-e8301d22fffe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:06.001978 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baee04a4-a856-4eb9-a936-e8301d22fffe None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:06.030390 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:06.051335 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96057abd-c767-4ff0-9def-feed77b7cbdd None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:06.094871 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8768d9f2-10ab-4c38-b840-6048ef843476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:06.095574 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8768d9f2-10ab-4c38-b840-6048ef843476 None admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:06.096262 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8768d9f2-10ab-4c38-b840-6048ef843476 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:06.096573 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8768d9f2-10ab-4c38-b840-6048ef843476 None admin] PATH_INFO: `/v3/services` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:06.097593 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8768d9f2-10ab-4c38-b840-6048ef843476 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:06.099139 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8768d9f2-10ab-4c38-b840-6048ef843476 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:06.117880 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 360/743] 10.222.0.42 () {60 vars in 1141 bytes} [Thu Jun 16 17:31:05 2022] POST /identity/v3/services => generated 240 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:31:06.118804 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8768d9f2-10ab-4c38-b840-6048ef843476 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:06.119605 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 384/744] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jun 16 17:31:05 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:31:06.126077 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baee04a4-a856-4eb9-a936-e8301d22fffe None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:06.283017 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:06.327233 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baee04a4-a856-4eb9-a936-e8301d22fffe None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:06.350094 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8768d9f2-10ab-4c38-b840-6048ef843476 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:06.380450 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daa1578-476e-4de5-af93-b8e34068825e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:06.381309 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daa1578-476e-4de5-af93-b8e34068825e None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:06.381592 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daa1578-476e-4de5-af93-b8e34068825e None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:06.381867 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daa1578-476e-4de5-af93-b8e34068825e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:06.395261 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daa1578-476e-4de5-af93-b8e34068825e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:06.395466 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3579d73-530a-4235-8d69-f925a8590df7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:06.396163 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3579d73-530a-4235-8d69-f925a8590df7 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:06.396486 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3579d73-530a-4235-8d69-f925a8590df7 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:06.396815 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daa1578-476e-4de5-af93-b8e34068825e None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:06.396919 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3579d73-530a-4235-8d69-f925a8590df7 None admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:06.397576 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3579d73-530a-4235-8d69-f925a8590df7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:06.397896 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3579d73-530a-4235-8d69-f925a8590df7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:06.399704 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3579d73-530a-4235-8d69-f925a8590df7 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:06.411393 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:06.418514 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-0daa1578-476e-4de5-af93-b8e34068825e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:31:06.418514 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:06.418514 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:31:06.418514 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:31:06.418514 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:31:06.418514 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:31:06.418514 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:31:06.418514 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:31:06.418514 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:31:06.418514 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:31:06.418514 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:31:06.418514 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:31:06.418514 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:31:06.418514 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:31:06.418514 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:31:06.419430 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:31:06.419430 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:31:06.419430 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:31:06.419430 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:31:06.419430 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:31:06.419430 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:31:06.419430 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:31:06.419430 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:31:06.419430 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:31:06.419430 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:31:06.419430 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:31:06.419430 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:31:06.419430 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:31:06.419430 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:31:06.419430 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:31:06.419430 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:31:06.419430 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:31:06.420343 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:31:06.420343 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:31:06.420343 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:31:06.420343 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:31:06.422003 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 385/745] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jun 16 17:31:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:31:06.424151 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 361/746] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jun 16 17:31:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:06.428062 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daa1578-476e-4de5-af93-b8e34068825e None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:06.429447 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3579d73-530a-4235-8d69-f925a8590df7 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:06.558104 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3579d73-530a-4235-8d69-f925a8590df7 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:06.582723 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daa1578-476e-4de5-af93-b8e34068825e None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:06.599887 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb099cea-495a-48b8-af08-b6d3a1c54c93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:06.601163 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb099cea-495a-48b8-af08-b6d3a1c54c93 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:06.601657 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb099cea-495a-48b8-af08-b6d3a1c54c93 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:06.602158 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb099cea-495a-48b8-af08-b6d3a1c54c93 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:06.612055 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb099cea-495a-48b8-af08-b6d3a1c54c93 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:06.614045 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb099cea-495a-48b8-af08-b6d3a1c54c93 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:06.631177 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-fb099cea-495a-48b8-af08-b6d3a1c54c93 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:31:06.631177 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:06.631177 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:31:06.631177 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:31:06.631177 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:31:06.631177 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:31:06.631177 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:31:06.631177 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:31:06.631177 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:31:06.631177 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:31:06.631177 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:31:06.631177 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:31:06.631177 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:31:06.631177 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:31:06.631177 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:31:06.632117 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:31:06.632117 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:31:06.632117 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:31:06.632117 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:31:06.632117 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:31:06.632117 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:31:06.632117 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:31:06.632117 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:31:06.632117 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:31:06.632117 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:31:06.632117 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:31:06.632117 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:31:06.632117 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:31:06.632117 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:31:06.632117 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:31:06.632117 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:31:06.632117 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:31:06.633056 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:31:06.633056 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:31:06.633056 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:31:06.633056 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:31:06.633056 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:31:06.633056 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:31:06.633056 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:31:06.633056 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:31:06.635604 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 362/747] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jun 16 17:31:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:31:06.643965 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb099cea-495a-48b8-af08-b6d3a1c54c93 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:06.663498 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ff55f7-7893-49fd-88ab-6df7499f8fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:06.664184 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ff55f7-7893-49fd-88ab-6df7499f8fe6 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:06.664516 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ff55f7-7893-49fd-88ab-6df7499f8fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:06.664828 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ff55f7-7893-49fd-88ab-6df7499f8fe6 None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:06.665562 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ff55f7-7893-49fd-88ab-6df7499f8fe6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:06.665876 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ff55f7-7893-49fd-88ab-6df7499f8fe6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:06.667783 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ff55f7-7893-49fd-88ab-6df7499f8fe6 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:06.671459 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:06.690322 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 386/748] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jun 16 17:31:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:06.695785 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ff55f7-7893-49fd-88ab-6df7499f8fe6 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:06.758464 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:06.833463 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb099cea-495a-48b8-af08-b6d3a1c54c93 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:06.881891 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ff55f7-7893-49fd-88ab-6df7499f8fe6 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:06.896465 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d21fd6-ef2e-4a61-8b16-5619d34f82de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:06.897145 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d21fd6-ef2e-4a61-8b16-5619d34f82de None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:06.897430 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d21fd6-ef2e-4a61-8b16-5619d34f82de None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:06.897710 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d21fd6-ef2e-4a61-8b16-5619d34f82de None admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:06.898504 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d21fd6-ef2e-4a61-8b16-5619d34f82de None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:06.898817 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d21fd6-ef2e-4a61-8b16-5619d34f82de None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:06.900275 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d21fd6-ef2e-4a61-8b16-5619d34f82de None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:06.916855 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 363/749] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Jun 16 17:31:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:31:06.923556 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d21fd6-ef2e-4a61-8b16-5619d34f82de None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:06.947464 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e69e4c5-7210-4774-9586-bb35de1f02ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:06.948653 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e69e4c5-7210-4774-9586-bb35de1f02ee None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:06.950122 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e69e4c5-7210-4774-9586-bb35de1f02ee None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:06.951967 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e69e4c5-7210-4774-9586-bb35de1f02ee None admin] PATH_INFO: `/v3/projects/service` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:06.968896 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e69e4c5-7210-4774-9586-bb35de1f02ee None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:06.974249 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e69e4c5-7210-4774-9586-bb35de1f02ee None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:07.008594 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-8e69e4c5-7210-4774-9586-bb35de1f02ee None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:31:07.008594 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:07.008594 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:31:07.008594 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:31:07.008594 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:31:07.008594 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:31:07.008594 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:31:07.008594 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:31:07.008594 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:31:07.008594 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:31:07.008594 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:31:07.008594 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:31:07.008594 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:31:07.008594 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:31:07.008594 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:31:07.009547 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:31:07.009547 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:31:07.009547 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:31:07.009547 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:31:07.009547 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:31:07.009547 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:31:07.009547 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:31:07.009547 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:31:07.009547 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:31:07.009547 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:31:07.009547 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:31:07.009547 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:31:07.009547 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:31:07.009547 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:31:07.009547 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:31:07.009547 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:31:07.009547 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:31:07.010440 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:31:07.010440 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:31:07.010440 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:31:07.010440 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:31:07.010440 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:31:07.010440 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:31:07.010440 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:31:07.010440 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:31:07.013875 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 387/750] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jun 16 17:31:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:31:07.020484 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e69e4c5-7210-4774-9586-bb35de1f02ee None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:07.130399 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d21fd6-ef2e-4a61-8b16-5619d34f82de None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:07.134441 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:07.168878 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:07.206037 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8f8522-a0d7-4f82-9095-baf5417283d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:07.206735 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8f8522-a0d7-4f82-9095-baf5417283d3 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:07.207024 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8f8522-a0d7-4f82-9095-baf5417283d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:07.207301 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8f8522-a0d7-4f82-9095-baf5417283d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:07.208203 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8f8522-a0d7-4f82-9095-baf5417283d3 None admin] RBAC: Adding query filter params (role.id=cd7dd29fcdc844c78544402dc30986fb, scope.project.id=8abb87f2f141451a8c6ed832db82be18, user.id=71443c30b4554719b5549493bf3cc00c) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:07.208580 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8f8522-a0d7-4f82-9095-baf5417283d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:07.211550 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8f8522-a0d7-4f82-9095-baf5417283d3 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:07.221086 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e69e4c5-7210-4774-9586-bb35de1f02ee None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:07.225602 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 364/751] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jun 16 17:31:06 2022] GET /identity/v3/role_assignments?user.id=71443c30b4554719b5549493bf3cc00c&scope.project.id=8abb87f2f141451a8c6ed832db82be18&role.id=cd7dd29fcdc844c78544402dc30986fb => generated 259 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:07.286908 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e1a6c3-68bf-4aa8-9b6f-0a1f42f09423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:07.287627 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e1a6c3-68bf-4aa8-9b6f-0a1f42f09423 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:07.287938 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e1a6c3-68bf-4aa8-9b6f-0a1f42f09423 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:07.288219 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e1a6c3-68bf-4aa8-9b6f-0a1f42f09423 None admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:07.289115 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e1a6c3-68bf-4aa8-9b6f-0a1f42f09423 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:07.289452 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e1a6c3-68bf-4aa8-9b6f-0a1f42f09423 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:07.291051 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e1a6c3-68bf-4aa8-9b6f-0a1f42f09423 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:07.310659 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 388/752] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Jun 16 17:31:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:07.317629 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8f8522-a0d7-4f82-9095-baf5417283d3 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:07.459961 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8f8522-a0d7-4f82-9095-baf5417283d3 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:07.503193 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f9dd86-e82b-4a41-ad2a-1f8b7e1a870c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:07.503904 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f9dd86-e82b-4a41-ad2a-1f8b7e1a870c None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:07.504190 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f9dd86-e82b-4a41-ad2a-1f8b7e1a870c None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:07.504556 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f9dd86-e82b-4a41-ad2a-1f8b7e1a870c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:07.505506 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f9dd86-e82b-4a41-ad2a-1f8b7e1a870c None admin] RBAC: Adding query filter params (role.id=3ca2dcbbd29d4a31a3ab78f2299bc239, scope.project.id=8abb87f2f141451a8c6ed832db82be18, user.id=47b7a9cd615c41d6983a0007b475038c) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:07.505847 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f9dd86-e82b-4a41-ad2a-1f8b7e1a870c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:07.507391 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f9dd86-e82b-4a41-ad2a-1f8b7e1a870c None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:07.514695 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 365/753] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jun 16 17:31:07 2022] GET /identity/v3/role_assignments?user.id=47b7a9cd615c41d6983a0007b475038c&scope.project.id=8abb87f2f141451a8c6ed832db82be18&role.id=3ca2dcbbd29d4a31a3ab78f2299bc239 => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:07.686526 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21e092b-eb94-4d90-82e4-1d1990bd6631 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:07.686895 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21e092b-eb94-4d90-82e4-1d1990bd6631 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:07.687790 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21e092b-eb94-4d90-82e4-1d1990bd6631 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:07.690004 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 389/754] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:31:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:31:07.696813 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c6aab1-1fcb-46de-9bb8-e5844d4aca16 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:07.697229 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c6aab1-1fcb-46de-9bb8-e5844d4aca16 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:07.697618 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c6aab1-1fcb-46de-9bb8-e5844d4aca16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:07.717368 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-c8c6aab1-1fcb-46de-9bb8-e5844d4aca16 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:31:07.718564 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-c8c6aab1-1fcb-46de-9bb8-e5844d4aca16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:31:07.772726 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:07.814897 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 366/755] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:31:07 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:31:07.838157 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21e092b-eb94-4d90-82e4-1d1990bd6631 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:07.928085 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086dc840-0245-4083-a523-24308328ee56 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:07.928085 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086dc840-0245-4083-a523-24308328ee56 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:07.928085 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086dc840-0245-4083-a523-24308328ee56 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:07.928673 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 367/756] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:31:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:31:07.933853 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7c7e00-828b-4629-be23-8f14e98095be None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:07.934414 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7c7e00-828b-4629-be23-8f14e98095be None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:07.934890 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7c7e00-828b-4629-be23-8f14e98095be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:07.965859 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21e092b-eb94-4d90-82e4-1d1990bd6631 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:07.973506 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-fa7c7e00-828b-4629-be23-8f14e98095be None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:31:07.974993 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-fa7c7e00-828b-4629-be23-8f14e98095be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:31:07.992779 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:08.003530 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9208d3a-3ca4-4c3a-bb8b-44584557c27e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:08.004259 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9208d3a-3ca4-4c3a-bb8b-44584557c27e None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:08.004586 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9208d3a-3ca4-4c3a-bb8b-44584557c27e None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:08.004869 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9208d3a-3ca4-4c3a-bb8b-44584557c27e None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:08.005531 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9208d3a-3ca4-4c3a-bb8b-44584557c27e None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:08.006906 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9208d3a-3ca4-4c3a-bb8b-44584557c27e None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:08.020942 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-e9208d3a-3ca4-4c3a-bb8b-44584557c27e None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 17:31:08.020942 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:08.020942 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 17:31:08.020942 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 17:31:08.020942 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 17:31:08.020942 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 17:31:08.020942 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 17:31:08.020942 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:31:08.020942 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 17:31:08.020942 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:31:08.020942 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:31:08.020942 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:31:08.020942 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:08.020942 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:31:08.020942 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:31:08.020942 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:31:08.021924 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:31:08.021924 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:31:08.021924 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:31:08.021924 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:31:08.021924 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:31:08.021924 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:31:08.021924 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:31:08.021924 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 17:31:08.021924 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 17:31:08.021924 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 17:31:08.021924 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 17:31:08.021924 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:31:08.021924 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:31:08.021924 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:31:08.021924 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:31:08.021924 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:31:08.022737 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:31:08.022737 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:31:08.022737 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:31:08.022737 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:31:08.022737 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:31:08.022737 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:31:08.022737 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:31:08.022737 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:31:08.022737 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:31:08.022737 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:31:08.022737 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:31:08.022737 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 17:31:08.022737 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:31:08.022737 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 17:31:08.022737 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:31:08.028452 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 390/757] 10.222.0.42 () {56 vars in 1129 bytes} [Thu Jun 16 17:31:07 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:31:08.029887 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9208d3a-3ca4-4c3a-bb8b-44584557c27e None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:08.126298 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 368/758] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:31:07 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 17:31:08.139230 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9208d3a-3ca4-4c3a-bb8b-44584557c27e None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:08.169270 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7c7e00-828b-4629-be23-8f14e98095be None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:08.181762 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ba08bd-fe84-463f-9144-20a080e3fd55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:08.182454 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ba08bd-fe84-463f-9144-20a080e3fd55 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:08.182869 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ba08bd-fe84-463f-9144-20a080e3fd55 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:08.183192 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ba08bd-fe84-463f-9144-20a080e3fd55 None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:08.183918 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ba08bd-fe84-463f-9144-20a080e3fd55 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:08.184247 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ba08bd-fe84-463f-9144-20a080e3fd55 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:08.185882 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ba08bd-fe84-463f-9144-20a080e3fd55 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:08.205547 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 391/759] 10.222.0.42 () {56 vars in 1124 bytes} [Thu Jun 16 17:31:08 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:08.209376 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ba08bd-fe84-463f-9144-20a080e3fd55 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:08.294996 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:08.373612 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ba08bd-fe84-463f-9144-20a080e3fd55 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:08.384905 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7c7e00-828b-4629-be23-8f14e98095be None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:08.396826 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:08.415984 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a72e8c3-2133-4151-b104-b2ff4d1ec568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:08.417174 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a72e8c3-2133-4151-b104-b2ff4d1ec568 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:08.417659 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a72e8c3-2133-4151-b104-b2ff4d1ec568 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:08.418384 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a72e8c3-2133-4151-b104-b2ff4d1ec568 None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:08.419287 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a72e8c3-2133-4151-b104-b2ff4d1ec568 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:08.419787 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a72e8c3-2133-4151-b104-b2ff4d1ec568 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:08.422389 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a72e8c3-2133-4151-b104-b2ff4d1ec568 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:08.428788 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 392/760] 10.222.0.42 () {56 vars in 1124 bytes} [Thu Jun 16 17:31:08 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:08.433960 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a72e8c3-2133-4151-b104-b2ff4d1ec568 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:08.458216 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd995a0-29f6-4dc7-993d-4479e6b89526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:08.459411 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd995a0-29f6-4dc7-993d-4479e6b89526 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:08.459921 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd995a0-29f6-4dc7-993d-4479e6b89526 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:08.460408 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd995a0-29f6-4dc7-993d-4479e6b89526 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:08.485598 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd995a0-29f6-4dc7-993d-4479e6b89526 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:31:08.500137 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-6bd995a0-29f6-4dc7-993d-4479e6b89526 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:31:08.500137 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:08.500137 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:31:08.500137 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:31:08.500137 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:31:08.500137 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:31:08.500137 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:31:08.500137 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:31:08.500137 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:31:08.500137 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:31:08.500137 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:31:08.500137 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:31:08.500137 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:31:08.500137 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:31:08.500137 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:31:08.500137 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:31:08.501191 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:31:08.501191 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:31:08.501191 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:31:08.501191 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:31:08.501191 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:31:08.501191 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:31:08.501191 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:31:08.501191 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:31:08.501191 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:31:08.501191 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:31:08.501191 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:31:08.501191 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:31:08.501191 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:31:08.501191 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:31:08.501191 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:31:08.501191 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:31:08.502022 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:31:08.502022 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:31:08.502022 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:31:08.502022 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:31:08.502022 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:31:08.502022 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:31:08.502022 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:31:08.502022 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:31:08.502022 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:31:08.502022 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:31:08.502022 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:31:08.502022 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:31:08.508384 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 369/761] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Jun 16 17:31:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:31:08.513239 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd995a0-29f6-4dc7-993d-4479e6b89526 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:08.670954 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd995a0-29f6-4dc7-993d-4479e6b89526 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:08.737315 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c886f39-57a1-4c23-8b5b-0c44f53b7598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:08.738032 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c886f39-57a1-4c23-8b5b-0c44f53b7598 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:08.738320 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c886f39-57a1-4c23-8b5b-0c44f53b7598 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:08.738597 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c886f39-57a1-4c23-8b5b-0c44f53b7598 None admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:08.739377 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c886f39-57a1-4c23-8b5b-0c44f53b7598 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:08.739689 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c886f39-57a1-4c23-8b5b-0c44f53b7598 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:08.741186 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c886f39-57a1-4c23-8b5b-0c44f53b7598 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:08.760564 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 370/762] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jun 16 17:31:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:08.766144 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c886f39-57a1-4c23-8b5b-0c44f53b7598 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:08.833790 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:08.848513 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a72e8c3-2133-4151-b104-b2ff4d1ec568 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:08.856117 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:08.894249 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ff60ae-46c8-44bb-8077-7d041a5bf89e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:08.895193 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ff60ae-46c8-44bb-8077-7d041a5bf89e None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:08.895696 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ff60ae-46c8-44bb-8077-7d041a5bf89e None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:08.896234 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ff60ae-46c8-44bb-8077-7d041a5bf89e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:08.897269 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ff60ae-46c8-44bb-8077-7d041a5bf89e None admin] RBAC: Adding query filter params (interface=public, service_id=7845609eacf2428aaec8a4a4e62a1246, region_id=RegionOne) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:08.897775 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ff60ae-46c8-44bb-8077-7d041a5bf89e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:08.899332 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ff60ae-46c8-44bb-8077-7d041a5bf89e None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:08.907686 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 393/763] 10.222.0.42 () {56 vars in 1251 bytes} [Thu Jun 16 17:31:08 2022] GET /identity/v3/endpoints?service_id=7845609eacf2428aaec8a4a4e62a1246&interface=public®ion_id=RegionOne => generated 194 bytes in 474 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:08.909312 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef928e87-bc02-4075-ba41-fd3f6aec2b47 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:08.909795 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef928e87-bc02-4075-ba41-fd3f6aec2b47 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:08.910262 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef928e87-bc02-4075-ba41-fd3f6aec2b47 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:08.912052 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 394/764] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:31:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:31:08.913008 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef928e87-bc02-4075-ba41-fd3f6aec2b47 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:08.928613 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c886f39-57a1-4c23-8b5b-0c44f53b7598 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:08.967709 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f9bdd7-2fe5-41de-a370-a6dae1b3339c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:08.968807 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f9bdd7-2fe5-41de-a370-a6dae1b3339c None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:08.969095 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f9bdd7-2fe5-41de-a370-a6dae1b3339c None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:08.969493 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f9bdd7-2fe5-41de-a370-a6dae1b3339c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:08.981783 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f9bdd7-2fe5-41de-a370-a6dae1b3339c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:08.983251 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f9bdd7-2fe5-41de-a370-a6dae1b3339c None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:09.002940 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-75f9bdd7-2fe5-41de-a370-a6dae1b3339c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:31:09.002940 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:09.002940 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:31:09.002940 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:31:09.002940 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:31:09.002940 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:31:09.002940 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:31:09.002940 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:31:09.002940 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:31:09.002940 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:31:09.002940 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:31:09.002940 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:31:09.002940 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:31:09.002940 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:31:09.002940 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:31:09.003870 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:31:09.003870 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:31:09.003870 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:31:09.003870 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:31:09.003870 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:31:09.003870 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:31:09.003870 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:31:09.003870 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:31:09.003870 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:31:09.003870 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:31:09.003870 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:31:09.003870 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:31:09.003870 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:31:09.003870 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:31:09.003870 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:31:09.003870 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:31:09.003870 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:31:09.004786 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:31:09.004786 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:31:09.004786 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:31:09.004786 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:31:09.008641 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 371/765] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jun 16 17:31:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:31:09.009961 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4913055-22bc-4600-93b6-206d2fe26dae None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:09.010249 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4913055-22bc-4600-93b6-206d2fe26dae None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:09.010524 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4913055-22bc-4600-93b6-206d2fe26dae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:09.029664 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-d4913055-22bc-4600-93b6-206d2fe26dae None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:31:09.031127 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-d4913055-22bc-4600-93b6-206d2fe26dae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:31:09.044637 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef928e87-bc02-4075-ba41-fd3f6aec2b47 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:09.091181 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f3e6a3-9c38-44bd-8c98-f2c6ad3b39f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:09.091756 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f3e6a3-9c38-44bd-8c98-f2c6ad3b39f7 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:09.092169 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f3e6a3-9c38-44bd-8c98-f2c6ad3b39f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:09.092574 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f3e6a3-9c38-44bd-8c98-f2c6ad3b39f7 None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:09.093675 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f3e6a3-9c38-44bd-8c98-f2c6ad3b39f7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:09.095400 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f3e6a3-9c38-44bd-8c98-f2c6ad3b39f7 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:09.102398 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 395/766] 10.222.0.42 () {56 vars in 1087 bytes} [Thu Jun 16 17:31:08 2022] GET /identity/v3/services => generated 772 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:09.103329 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f3e6a3-9c38-44bd-8c98-f2c6ad3b39f7 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:09.126631 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 372/767] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:31:09 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:31:09.145339 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:09.151424 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4913055-22bc-4600-93b6-206d2fe26dae None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:09.164509 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:09.224678 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f3e6a3-9c38-44bd-8c98-f2c6ad3b39f7 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:09.263819 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b35ddb-d8a1-4ddd-9ef6-3cb77d357a9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:09.264549 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b35ddb-d8a1-4ddd-9ef6-3cb77d357a9c None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:09.264831 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b35ddb-d8a1-4ddd-9ef6-3cb77d357a9c None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:09.266184 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b35ddb-d8a1-4ddd-9ef6-3cb77d357a9c None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:09.266961 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b35ddb-d8a1-4ddd-9ef6-3cb77d357a9c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:09.267271 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b35ddb-d8a1-4ddd-9ef6-3cb77d357a9c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:09.268760 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b35ddb-d8a1-4ddd-9ef6-3cb77d357a9c None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:09.282325 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 396/768] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jun 16 17:31:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:09.284149 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4913055-22bc-4600-93b6-206d2fe26dae None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:09.290077 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b35ddb-d8a1-4ddd-9ef6-3cb77d357a9c None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:09.332840 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd29db8-37f0-43da-a90a-b1da1afc42d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:09.333661 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd29db8-37f0-43da-a90a-b1da1afc42d5 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:09.333945 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd29db8-37f0-43da-a90a-b1da1afc42d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:09.334216 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd29db8-37f0-43da-a90a-b1da1afc42d5 None admin] PATH_INFO: `/v3/services/network` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:09.335466 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd29db8-37f0-43da-a90a-b1da1afc42d5 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:09.337424 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd29db8-37f0-43da-a90a-b1da1afc42d5 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:09.351505 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-1dd29db8-37f0-43da-a90a-b1da1afc42d5 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 17:31:09.351505 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:09.351505 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 17:31:09.351505 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 17:31:09.351505 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 17:31:09.351505 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 17:31:09.351505 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 17:31:09.351505 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:31:09.351505 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 17:31:09.351505 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:31:09.351505 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:31:09.351505 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:31:09.351505 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:09.351505 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:31:09.351505 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:31:09.351505 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:31:09.351505 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:31:09.352600 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:31:09.352600 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:31:09.352600 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:31:09.352600 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:31:09.352600 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:31:09.352600 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:31:09.352600 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 17:31:09.352600 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 17:31:09.352600 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 17:31:09.352600 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 17:31:09.352600 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:31:09.352600 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:31:09.352600 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:31:09.352600 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:31:09.352600 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:31:09.352600 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:31:09.353508 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:31:09.353508 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:31:09.353508 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:31:09.353508 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:31:09.353508 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:31:09.353508 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:31:09.353508 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:31:09.353508 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:31:09.353508 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:31:09.353508 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:31:09.353508 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 17:31:09.353508 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:31:09.353508 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 17:31:09.353508 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:31:09.353508 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 373/769] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jun 16 17:31:09 2022] GET /identity/v3/services/network => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:31:09.359689 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd29db8-37f0-43da-a90a-b1da1afc42d5 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:09.444927 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:09.454904 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b35ddb-d8a1-4ddd-9ef6-3cb77d357a9c None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:09.458821 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:09.505314 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd29db8-37f0-43da-a90a-b1da1afc42d5 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:09.505925 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e884797c-dc44-4280-9cae-9855e3e0efa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:09.506395 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e884797c-dc44-4280-9cae-9855e3e0efa1 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:09.506676 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e884797c-dc44-4280-9cae-9855e3e0efa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:09.506953 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e884797c-dc44-4280-9cae-9855e3e0efa1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:09.517119 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e884797c-dc44-4280-9cae-9855e3e0efa1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:09.518571 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e884797c-dc44-4280-9cae-9855e3e0efa1 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:09.533895 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-e884797c-dc44-4280-9cae-9855e3e0efa1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 17:31:09.533895 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:09.533895 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:31:09.533895 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:31:09.533895 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:31:09.533895 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:31:09.533895 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:31:09.533895 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:31:09.533895 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:31:09.533895 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:31:09.533895 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:31:09.533895 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:31:09.533895 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:31:09.533895 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:31:09.533895 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:31:09.533895 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:31:09.534865 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:31:09.534865 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:31:09.534865 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:31:09.534865 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:31:09.534865 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:31:09.534865 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:31:09.534865 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:31:09.534865 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:31:09.534865 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:31:09.534865 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:31:09.534865 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:31:09.534865 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:31:09.534865 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:31:09.534865 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:31:09.534865 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:31:09.534865 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:31:09.534865 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:31:09.534865 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:31:09.535785 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:31:09.535785 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:31:09.535785 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:31:09.535785 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:31:09.535785 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:31:09.535785 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:31:09.535785 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:31:09.535785 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:31:09.535785 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 17:31:09.535785 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:31:09.535785 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 397/770] 10.222.0.42 () {56 vars in 1093 bytes} [Thu Jun 16 17:31:09 2022] GET /identity/v3/users/nova => generated 82 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:31:09.549244 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316b3762-6a3f-4bc2-9433-1b491ebf5c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:09.549929 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316b3762-6a3f-4bc2-9433-1b491ebf5c9c None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:09.550215 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316b3762-6a3f-4bc2-9433-1b491ebf5c9c None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:09.550492 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316b3762-6a3f-4bc2-9433-1b491ebf5c9c None admin] PATH_INFO: `/v3/services` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:09.551201 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316b3762-6a3f-4bc2-9433-1b491ebf5c9c None admin] RBAC: Adding query filter params (name=network) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:09.551502 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316b3762-6a3f-4bc2-9433-1b491ebf5c9c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:09.553356 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316b3762-6a3f-4bc2-9433-1b491ebf5c9c None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:09.553989 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e884797c-dc44-4280-9cae-9855e3e0efa1 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:09.558645 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 374/771] 10.222.0.42 () {56 vars in 1112 bytes} [Thu Jun 16 17:31:09 2022] GET /identity/v3/services?name=network => generated 124 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:09.564716 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316b3762-6a3f-4bc2-9433-1b491ebf5c9c None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:09.686704 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316b3762-6a3f-4bc2-9433-1b491ebf5c9c None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:09.728060 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ff96d8-f9c6-4446-b0cc-23ed8b37e9b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:09.728812 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ff96d8-f9c6-4446-b0cc-23ed8b37e9b5 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:09.729099 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ff96d8-f9c6-4446-b0cc-23ed8b37e9b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:09.729374 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ff96d8-f9c6-4446-b0cc-23ed8b37e9b5 None admin] PATH_INFO: `/v3/services` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:09.730086 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ff96d8-f9c6-4446-b0cc-23ed8b37e9b5 None admin] RBAC: Adding query filter params (type=network) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:09.730393 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ff96d8-f9c6-4446-b0cc-23ed8b37e9b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:09.731788 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ff96d8-f9c6-4446-b0cc-23ed8b37e9b5 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:09.739586 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 375/772] 10.222.0.42 () {56 vars in 1112 bytes} [Thu Jun 16 17:31:09 2022] GET /identity/v3/services?type=network => generated 350 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:31:09.745159 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ff96d8-f9c6-4446-b0cc-23ed8b37e9b5 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:09.748138 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:09.784049 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e884797c-dc44-4280-9cae-9855e3e0efa1 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:09.858938 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4827f09e-6e52-45b2-8ec2-1ff6a499f177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:09.858938 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4827f09e-6e52-45b2-8ec2-1ff6a499f177 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:09.859259 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4827f09e-6e52-45b2-8ec2-1ff6a499f177 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:09.860988 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4827f09e-6e52-45b2-8ec2-1ff6a499f177 None admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:09.860988 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4827f09e-6e52-45b2-8ec2-1ff6a499f177 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:09.860988 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4827f09e-6e52-45b2-8ec2-1ff6a499f177 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:09.862341 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4827f09e-6e52-45b2-8ec2-1ff6a499f177 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:09.892994 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 398/773] 10.222.0.42 () {56 vars in 1133 bytes} [Thu Jun 16 17:31:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:09.894610 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b74341-fd19-4c4a-b0c2-4efff13e11ad None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:09.894903 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b74341-fd19-4c4a-b0c2-4efff13e11ad None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:09.895200 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b74341-fd19-4c4a-b0c2-4efff13e11ad None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:09.896564 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 399/774] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:31:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:31:09.901283 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b74341-fd19-4c4a-b0c2-4efff13e11ad None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:09.901571 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ff96d8-f9c6-4446-b0cc-23ed8b37e9b5 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:09.917508 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:09.947688 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f080c0-384b-4f0a-8358-44f31ea3f925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:09.948361 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f080c0-384b-4f0a-8358-44f31ea3f925 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:09.948675 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f080c0-384b-4f0a-8358-44f31ea3f925 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:09.948954 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f080c0-384b-4f0a-8358-44f31ea3f925 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:09.949754 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f080c0-384b-4f0a-8358-44f31ea3f925 None admin] RBAC: Adding query filter params (interface=public, service_id=1f40dc275a574d599fa5034bb4b5832e, region_id=RegionOne) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:09.950068 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f080c0-384b-4f0a-8358-44f31ea3f925 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:09.951544 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f080c0-384b-4f0a-8358-44f31ea3f925 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:09.961053 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 376/775] 10.222.0.42 () {56 vars in 1251 bytes} [Thu Jun 16 17:31:09 2022] GET /identity/v3/endpoints?service_id=1f40dc275a574d599fa5034bb4b5832e&interface=public®ion_id=RegionOne => generated 194 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:31:09.962610 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eec61b0-1a0a-4d44-ac22-c8bc7db563a6 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:09.962901 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eec61b0-1a0a-4d44-ac22-c8bc7db563a6 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:09.963183 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eec61b0-1a0a-4d44-ac22-c8bc7db563a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:09.982823 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-8eec61b0-1a0a-4d44-ac22-c8bc7db563a6 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:31:09.985274 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-8eec61b0-1a0a-4d44-ac22-c8bc7db563a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:31:10.018249 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b74341-fd19-4c4a-b0c2-4efff13e11ad None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:10.058845 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:10.061106 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df54681-4dda-445f-b49e-269e8b15eecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:10.063624 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df54681-4dda-445f-b49e-269e8b15eecd None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:10.064068 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df54681-4dda-445f-b49e-269e8b15eecd None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:10.064571 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df54681-4dda-445f-b49e-269e8b15eecd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:10.068485 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 377/776] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:31:09 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:31:10.069456 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eec61b0-1a0a-4d44-ac22-c8bc7db563a6 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:10.076559 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df54681-4dda-445f-b49e-269e8b15eecd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:10.078972 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df54681-4dda-445f-b49e-269e8b15eecd None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:10.099526 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-5df54681-4dda-445f-b49e-269e8b15eecd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:31:10.099526 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:10.099526 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:31:10.099526 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:31:10.099526 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:31:10.099526 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:31:10.099526 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:31:10.099526 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:31:10.099526 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:31:10.099526 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:31:10.099526 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:31:10.099526 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:31:10.099526 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:31:10.099526 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:31:10.099526 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:31:10.100531 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:31:10.100531 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:31:10.100531 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:31:10.100531 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:31:10.100531 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:31:10.100531 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:31:10.100531 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:31:10.100531 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:31:10.100531 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:31:10.100531 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:31:10.100531 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:31:10.100531 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:31:10.100531 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:31:10.100531 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:31:10.100531 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:31:10.100531 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:31:10.100531 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:31:10.101434 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:31:10.101434 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:31:10.101434 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:31:10.101434 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:31:10.103955 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 400/777] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jun 16 17:31:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:31:10.105078 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df54681-4dda-445f-b49e-269e8b15eecd None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:10.177682 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:10.184225 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eec61b0-1a0a-4d44-ac22-c8bc7db563a6 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:10.223924 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df54681-4dda-445f-b49e-269e8b15eecd None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:10.224597 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bfd33a-a307-4615-b3a7-b63c863ef047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:10.225585 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bfd33a-a307-4615-b3a7-b63c863ef047 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:10.226000 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bfd33a-a307-4615-b3a7-b63c863ef047 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:10.226355 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bfd33a-a307-4615-b3a7-b63c863ef047 None admin] PATH_INFO: `/v3/services` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:10.227416 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bfd33a-a307-4615-b3a7-b63c863ef047 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:10.229060 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bfd33a-a307-4615-b3a7-b63c863ef047 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:10.235359 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 378/778] 10.222.0.42 () {56 vars in 1087 bytes} [Thu Jun 16 17:31:10 2022] GET /identity/v3/services => generated 772 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:10.236605 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bfd33a-a307-4615-b3a7-b63c863ef047 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:10.268794 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8c63a1-db84-49cb-820f-31d3e0bf1725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:10.270122 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8c63a1-db84-49cb-820f-31d3e0bf1725 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:10.270715 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8c63a1-db84-49cb-820f-31d3e0bf1725 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:10.271222 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8c63a1-db84-49cb-820f-31d3e0bf1725 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:10.284329 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8c63a1-db84-49cb-820f-31d3e0bf1725 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:31:10.293270 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-7e8c63a1-db84-49cb-820f-31d3e0bf1725 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:31:10.293270 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:10.293270 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:31:10.293270 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:31:10.293270 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:31:10.293270 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:31:10.293270 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:31:10.293270 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:31:10.293270 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:31:10.293270 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:31:10.293270 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:31:10.293270 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:31:10.293270 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:31:10.293270 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:31:10.293270 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:31:10.293270 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:31:10.294280 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:31:10.294280 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:31:10.294280 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:31:10.294280 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:31:10.294280 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:31:10.294280 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:31:10.294280 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:31:10.294280 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:31:10.294280 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:31:10.294280 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:31:10.294280 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:31:10.294280 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:31:10.294280 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:31:10.294280 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:31:10.294280 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:31:10.294280 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:31:10.295118 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:31:10.295118 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:31:10.295118 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:31:10.295118 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:31:10.295118 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:31:10.295118 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:31:10.295118 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:31:10.295118 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:31:10.295118 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:31:10.295118 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:31:10.295118 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:31:10.295118 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:31:10.298297 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 401/779] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Jun 16 17:31:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:31:10.305617 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8c63a1-db84-49cb-820f-31d3e0bf1725 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:10.311796 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:10.398162 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bfd33a-a307-4615-b3a7-b63c863ef047 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:10.477481 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd6f9e4-688a-4841-b355-33bab0b7ec16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:10.478164 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd6f9e4-688a-4841-b355-33bab0b7ec16 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:10.478443 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd6f9e4-688a-4841-b355-33bab0b7ec16 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:10.478714 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd6f9e4-688a-4841-b355-33bab0b7ec16 None admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:10.479452 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd6f9e4-688a-4841-b355-33bab0b7ec16 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:10.479750 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd6f9e4-688a-4841-b355-33bab0b7ec16 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:10.482018 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd6f9e4-688a-4841-b355-33bab0b7ec16 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:10.498409 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 379/780] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jun 16 17:31:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:10.506302 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd6f9e4-688a-4841-b355-33bab0b7ec16 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:10.516068 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8c63a1-db84-49cb-820f-31d3e0bf1725 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:10.544236 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:10.561189 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfbce3e-76e3-49dc-a038-7b895a7de36a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:10.563073 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfbce3e-76e3-49dc-a038-7b895a7de36a None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:10.563519 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfbce3e-76e3-49dc-a038-7b895a7de36a None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:10.564214 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfbce3e-76e3-49dc-a038-7b895a7de36a None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:10.565001 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfbce3e-76e3-49dc-a038-7b895a7de36a None admin] RBAC: Adding query filter params (name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:10.565315 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfbce3e-76e3-49dc-a038-7b895a7de36a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:10.566700 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfbce3e-76e3-49dc-a038-7b895a7de36a None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:10.580106 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 402/781] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Jun 16 17:31:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:31:10.581714 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dc63a8-3ff5-451f-ae2a-b0434c12c6e7 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:10.582063 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dc63a8-3ff5-451f-ae2a-b0434c12c6e7 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:10.582341 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dc63a8-3ff5-451f-ae2a-b0434c12c6e7 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:10.584000 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 403/782] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:31:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:31:10.588780 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dc63a8-3ff5-451f-ae2a-b0434c12c6e7 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:10.645886 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd6f9e4-688a-4841-b355-33bab0b7ec16 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:10.672300 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:10.693611 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4caea3-1ce7-4960-a347-084fddc4cad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:10.694985 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4caea3-1ce7-4960-a347-084fddc4cad7 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:10.695285 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4caea3-1ce7-4960-a347-084fddc4cad7 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:10.695568 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4caea3-1ce7-4960-a347-084fddc4cad7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:10.706303 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4caea3-1ce7-4960-a347-084fddc4cad7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:10.707777 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4caea3-1ce7-4960-a347-084fddc4cad7 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:10.709637 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dc63a8-3ff5-451f-ae2a-b0434c12c6e7 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:10.729219 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-9e4caea3-1ce7-4960-a347-084fddc4cad7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:31:10.729219 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:10.729219 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:31:10.729219 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:31:10.729219 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:31:10.729219 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:31:10.729219 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:31:10.729219 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:31:10.729219 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:31:10.729219 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:31:10.729219 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:31:10.729219 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:31:10.729219 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:31:10.729219 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:31:10.729219 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:31:10.730137 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:31:10.730137 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:31:10.730137 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:31:10.730137 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:31:10.730137 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:31:10.730137 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:31:10.730137 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:31:10.730137 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:31:10.730137 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:31:10.730137 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:31:10.730137 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:31:10.730137 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:31:10.730137 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:31:10.730137 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:31:10.730137 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:31:10.730137 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:31:10.730137 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:31:10.731047 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:31:10.731047 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:31:10.731047 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:31:10.731047 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:31:10.731047 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:31:10.731047 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:31:10.731047 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:31:10.731047 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:31:10.733594 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 380/783] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jun 16 17:31:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:31:10.735799 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e65689-f5de-49b7-9e0c-159bc6ae5137 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:10.736299 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e65689-f5de-49b7-9e0c-159bc6ae5137 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:10.736806 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e65689-f5de-49b7-9e0c-159bc6ae5137 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:10.754270 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990c61c6-5a4e-4cd6-8ec9-8ca40722ebf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:10.755449 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990c61c6-5a4e-4cd6-8ec9-8ca40722ebf9 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:10.755983 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990c61c6-5a4e-4cd6-8ec9-8ca40722ebf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:10.756479 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990c61c6-5a4e-4cd6-8ec9-8ca40722ebf9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:10.757611 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-91e65689-f5de-49b7-9e0c-159bc6ae5137 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:31:10.759082 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-91e65689-f5de-49b7-9e0c-159bc6ae5137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:31:10.768913 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990c61c6-5a4e-4cd6-8ec9-8ca40722ebf9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:10.770772 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990c61c6-5a4e-4cd6-8ec9-8ca40722ebf9 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:10.789966 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-990c61c6-5a4e-4cd6-8ec9-8ca40722ebf9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:31:10.789966 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:10.789966 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:31:10.789966 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:31:10.789966 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:31:10.789966 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:31:10.789966 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:31:10.789966 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:31:10.789966 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:31:10.789966 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:31:10.789966 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:31:10.789966 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:31:10.789966 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:31:10.789966 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:31:10.789966 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:31:10.790900 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:31:10.790900 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:31:10.790900 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:31:10.790900 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:31:10.790900 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:31:10.790900 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:31:10.790900 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:31:10.790900 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:31:10.790900 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:31:10.790900 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:31:10.790900 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:31:10.790900 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:31:10.790900 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:31:10.790900 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:31:10.790900 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:31:10.790900 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:31:10.790900 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:31:10.791817 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:31:10.791817 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:31:10.791817 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:31:10.791817 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:31:10.794031 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 404/784] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jun 16 17:31:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:31:10.795024 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990c61c6-5a4e-4cd6-8ec9-8ca40722ebf9 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:10.843025 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 381/785] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:31:10 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:31:10.847731 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:10.849820 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e65689-f5de-49b7-9e0c-159bc6ae5137 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:10.944709 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990c61c6-5a4e-4cd6-8ec9-8ca40722ebf9 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:10.989817 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1b6527-7894-48ba-ad1e-0feba1582100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:10.990812 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1b6527-7894-48ba-ad1e-0feba1582100 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:10.991102 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1b6527-7894-48ba-ad1e-0feba1582100 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:10.991379 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1b6527-7894-48ba-ad1e-0feba1582100 None admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:10.992501 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1b6527-7894-48ba-ad1e-0feba1582100 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:10.992816 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1b6527-7894-48ba-ad1e-0feba1582100 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:10.994805 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1b6527-7894-48ba-ad1e-0feba1582100 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:10.997670 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e65689-f5de-49b7-9e0c-159bc6ae5137 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:11.014736 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 405/786] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Jun 16 17:31:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:11.014736 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1b6527-7894-48ba-ad1e-0feba1582100 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:11.018242 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:11.051650 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3f2cdb-a139-4383-8ecf-6e87e5c363a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:11.052577 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3f2cdb-a139-4383-8ecf-6e87e5c363a4 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:11.052961 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3f2cdb-a139-4383-8ecf-6e87e5c363a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:11.053339 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3f2cdb-a139-4383-8ecf-6e87e5c363a4 None admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:11.054169 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3f2cdb-a139-4383-8ecf-6e87e5c363a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:11.054605 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3f2cdb-a139-4383-8ecf-6e87e5c363a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:11.056973 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3f2cdb-a139-4383-8ecf-6e87e5c363a4 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:11.074302 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 382/787] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jun 16 17:31:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:11.074302 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3f2cdb-a139-4383-8ecf-6e87e5c363a4 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:11.129199 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:11.133453 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1b6527-7894-48ba-ad1e-0feba1582100 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:11.173124 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa53f8d6-036f-416a-8487-7d5c3f226f8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:11.174421 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa53f8d6-036f-416a-8487-7d5c3f226f8b None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:11.174939 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa53f8d6-036f-416a-8487-7d5c3f226f8b None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:11.175392 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa53f8d6-036f-416a-8487-7d5c3f226f8b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:11.187819 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa53f8d6-036f-416a-8487-7d5c3f226f8b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:31:11.188496 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3f2cdb-a139-4383-8ecf-6e87e5c363a4 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:11.197070 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-aa53f8d6-036f-416a-8487-7d5c3f226f8b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:31:11.197070 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:11.197070 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:31:11.197070 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:31:11.197070 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:31:11.197070 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:31:11.197070 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:31:11.197070 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:31:11.197070 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:31:11.197070 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:31:11.197070 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:31:11.197070 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:31:11.197070 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:31:11.197070 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:31:11.197070 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:31:11.197070 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:31:11.198040 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:31:11.198040 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:31:11.198040 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:31:11.198040 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:31:11.198040 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:31:11.198040 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:31:11.198040 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:31:11.198040 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:31:11.198040 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:31:11.198040 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:31:11.198040 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:31:11.198040 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:31:11.198040 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:31:11.198040 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:31:11.198040 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:31:11.198040 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:31:11.198880 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:31:11.198880 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:31:11.198880 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:31:11.198880 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:31:11.198880 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:31:11.198880 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:31:11.198880 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:31:11.198880 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:31:11.198880 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:31:11.198880 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:31:11.198880 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:31:11.198880 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:31:11.204919 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 406/788] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Jun 16 17:31:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:31:11.206196 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa53f8d6-036f-416a-8487-7d5c3f226f8b None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:11.226860 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4306616d-1a1a-4c98-a927-401f5ebb8058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:11.227634 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4306616d-1a1a-4c98-a927-401f5ebb8058 None admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:11.228014 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4306616d-1a1a-4c98-a927-401f5ebb8058 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:11.228402 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4306616d-1a1a-4c98-a927-401f5ebb8058 None admin] PATH_INFO: `/v3/projects/8abb87f2f141451a8c6ed832db82be18/users/f86c16179a70474a817be9fda33ece9d/roles/3ca2dcbbd29d4a31a3ab78f2299bc239` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:11.230629 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4306616d-1a1a-4c98-a927-401f5ebb8058 None admin] RBAC: Authorizing `identity:create_grant(project_id=8abb87f2f141451a8c6ed832db82be18, user_id=f86c16179a70474a817be9fda33ece9d, role_id=3ca2dcbbd29d4a31a3ab78f2299bc239)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:11.232271 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4306616d-1a1a-4c98-a927-401f5ebb8058 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:11.252594 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 383/789] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Jun 16 17:31:11 2022] PUT /identity/v3/projects/8abb87f2f141451a8c6ed832db82be18/users/f86c16179a70474a817be9fda33ece9d/roles/3ca2dcbbd29d4a31a3ab78f2299bc239 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:31:11.252594 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4306616d-1a1a-4c98-a927-401f5ebb8058 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:11.287065 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:11.378470 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4306616d-1a1a-4c98-a927-401f5ebb8058 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:11.393114 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa53f8d6-036f-416a-8487-7d5c3f226f8b None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:11.424106 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bedfa3-d7c8-4eb5-a218-3678010484e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:11.425125 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bedfa3-d7c8-4eb5-a218-3678010484e1 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:11.425513 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bedfa3-d7c8-4eb5-a218-3678010484e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:11.425887 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bedfa3-d7c8-4eb5-a218-3678010484e1 None admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:11.426749 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bedfa3-d7c8-4eb5-a218-3678010484e1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:11.429949 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bedfa3-d7c8-4eb5-a218-3678010484e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:11.429949 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bedfa3-d7c8-4eb5-a218-3678010484e1 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:11.440334 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 384/790] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jun 16 17:31:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:11.446778 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bedfa3-d7c8-4eb5-a218-3678010484e1 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:11.489801 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d0a5a8-06dc-484c-819d-8624d30b4220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:11.490649 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d0a5a8-06dc-484c-819d-8624d30b4220 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:11.490922 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d0a5a8-06dc-484c-819d-8624d30b4220 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:11.491209 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d0a5a8-06dc-484c-819d-8624d30b4220 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:11.513362 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d0a5a8-06dc-484c-819d-8624d30b4220 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:11.514875 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d0a5a8-06dc-484c-819d-8624d30b4220 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:11.545063 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-d5d0a5a8-06dc-484c-819d-8624d30b4220 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 17:31:11.545063 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:11.545063 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:31:11.545063 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:31:11.545063 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:31:11.545063 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:31:11.545063 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:31:11.545063 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:31:11.545063 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:31:11.545063 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:31:11.545063 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:31:11.545063 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:31:11.545063 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:31:11.545063 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:31:11.545063 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:31:11.545063 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:31:11.546060 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:31:11.546060 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:31:11.546060 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:31:11.546060 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:31:11.546060 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:31:11.546060 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:31:11.546060 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:31:11.546060 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:31:11.546060 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:31:11.546060 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:31:11.546060 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:31:11.546060 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:31:11.546060 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:31:11.546060 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:31:11.546060 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:31:11.546060 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:31:11.546060 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:31:11.546060 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:31:11.547664 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:31:11.547664 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:31:11.547664 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:31:11.547664 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:31:11.547664 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:31:11.547664 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:31:11.547664 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:31:11.547664 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:31:11.547664 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 17:31:11.547664 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:31:11.552822 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 407/791] 10.222.0.42 () {56 vars in 1117 bytes} [Thu Jun 16 17:31:11 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:31:11.558202 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:11.561452 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d0a5a8-06dc-484c-819d-8624d30b4220 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:11.567868 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bedfa3-d7c8-4eb5-a218-3678010484e1 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:11.600170 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:11.615476 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159164bc-9a3f-4976-9d4f-36f993b182d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:11.616404 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159164bc-9a3f-4976-9d4f-36f993b182d1 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:11.616711 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159164bc-9a3f-4976-9d4f-36f993b182d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:11.616988 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159164bc-9a3f-4976-9d4f-36f993b182d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:11.630198 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159164bc-9a3f-4976-9d4f-36f993b182d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:11.631799 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159164bc-9a3f-4976-9d4f-36f993b182d1 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:11.651377 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-159164bc-9a3f-4976-9d4f-36f993b182d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:31:11.651377 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:11.651377 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:31:11.651377 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:31:11.651377 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:31:11.651377 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:31:11.651377 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:31:11.651377 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:31:11.651377 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:31:11.651377 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:31:11.651377 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:31:11.651377 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:31:11.651377 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:31:11.651377 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:31:11.651377 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:31:11.652295 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:31:11.652295 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:31:11.652295 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:31:11.652295 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:31:11.652295 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:31:11.652295 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:31:11.652295 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:31:11.652295 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:31:11.652295 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:31:11.652295 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:31:11.652295 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:31:11.652295 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:31:11.652295 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:31:11.652295 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:31:11.652295 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:31:11.652295 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:31:11.652295 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:31:11.653657 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:31:11.653657 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:31:11.653657 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:31:11.653657 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:31:11.653657 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 385/792] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jun 16 17:31:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:31:11.659508 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159164bc-9a3f-4976-9d4f-36f993b182d1 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:11.695678 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d0a5a8-06dc-484c-819d-8624d30b4220 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:11.735231 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c23f96-a395-4a6d-b69d-6821b27bc52b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:11.735908 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c23f96-a395-4a6d-b69d-6821b27bc52b None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:11.736190 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c23f96-a395-4a6d-b69d-6821b27bc52b None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:11.736492 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c23f96-a395-4a6d-b69d-6821b27bc52b None admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:11.737502 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c23f96-a395-4a6d-b69d-6821b27bc52b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:11.737816 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c23f96-a395-4a6d-b69d-6821b27bc52b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:11.739309 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c23f96-a395-4a6d-b69d-6821b27bc52b None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:11.759897 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 408/793] 10.222.0.42 () {56 vars in 1149 bytes} [Thu Jun 16 17:31:11 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:11.761309 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93a312f-916f-4a8d-9a8f-71e51173de7a None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:11.761596 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93a312f-916f-4a8d-9a8f-71e51173de7a None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:11.761872 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93a312f-916f-4a8d-9a8f-71e51173de7a None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:11.763366 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 409/794] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:31:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:31:11.764660 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93a312f-916f-4a8d-9a8f-71e51173de7a None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:11.784818 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159164bc-9a3f-4976-9d4f-36f993b182d1 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:11.825169 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941335ce-377f-4410-9a2e-3d111ea0e350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:11.826017 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941335ce-377f-4410-9a2e-3d111ea0e350 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:11.826465 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941335ce-377f-4410-9a2e-3d111ea0e350 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:11.826806 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941335ce-377f-4410-9a2e-3d111ea0e350 None admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:11.828251 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941335ce-377f-4410-9a2e-3d111ea0e350 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:11.828701 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941335ce-377f-4410-9a2e-3d111ea0e350 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:11.830102 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941335ce-377f-4410-9a2e-3d111ea0e350 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:11.847761 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 386/795] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jun 16 17:31:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:11.849427 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bfe577-5322-4cb2-8961-2b93b95bf275 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:11.849723 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bfe577-5322-4cb2-8961-2b93b95bf275 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:11.850007 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bfe577-5322-4cb2-8961-2b93b95bf275 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:11.856125 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:11.857733 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:11.877347 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-43bfe577-5322-4cb2-8961-2b93b95bf275 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:31:11.879614 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-43bfe577-5322-4cb2-8961-2b93b95bf275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:31:11.885743 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93a312f-916f-4a8d-9a8f-71e51173de7a None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:11.927511 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0564973-8ef2-413e-bade-60c4d9bfcf9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:11.928734 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0564973-8ef2-413e-bade-60c4d9bfcf9e None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:11.929159 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0564973-8ef2-413e-bade-60c4d9bfcf9e None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:11.929569 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0564973-8ef2-413e-bade-60c4d9bfcf9e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:11.941700 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0564973-8ef2-413e-bade-60c4d9bfcf9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:11.943471 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0564973-8ef2-413e-bade-60c4d9bfcf9e None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:11.958169 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 387/796] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:31:11 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:31:11.959131 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bfe577-5322-4cb2-8961-2b93b95bf275 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:11.963831 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-e0564973-8ef2-413e-bade-60c4d9bfcf9e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:31:11.963831 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:11.963831 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:31:11.963831 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:31:11.963831 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:31:11.963831 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:31:11.963831 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:31:11.963831 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:31:11.963831 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:31:11.963831 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:31:11.963831 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:31:11.963831 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:31:11.963831 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:31:11.963831 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:31:11.963831 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:31:11.964750 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:31:11.964750 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:31:11.964750 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:31:11.964750 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:31:11.964750 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:31:11.964750 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:31:11.964750 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:31:11.964750 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:31:11.964750 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:31:11.964750 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:31:11.964750 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:31:11.964750 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:31:11.964750 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:31:11.964750 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:31:11.964750 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:31:11.964750 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:31:11.964750 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:31:11.965656 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:31:11.965656 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:31:11.965656 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:31:11.965656 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:31:11.967903 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 410/797] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jun 16 17:31:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:31:11.974087 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0564973-8ef2-413e-bade-60c4d9bfcf9e None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:12.075475 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bfe577-5322-4cb2-8961-2b93b95bf275 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:12.094537 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0564973-8ef2-413e-bade-60c4d9bfcf9e None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:12.122874 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:12.125321 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21986ac0-73b2-477a-bdc1-76ec6aadb690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:12.128410 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21986ac0-73b2-477a-bdc1-76ec6aadb690 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:12.129370 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21986ac0-73b2-477a-bdc1-76ec6aadb690 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:12.129864 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21986ac0-73b2-477a-bdc1-76ec6aadb690 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:12.132246 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:12.133245 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0eaa2b5-8faa-4568-a92d-75c482c20127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:12.133919 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eaa2b5-8faa-4568-a92d-75c482c20127 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:12.134205 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eaa2b5-8faa-4568-a92d-75c482c20127 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:12.134481 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eaa2b5-8faa-4568-a92d-75c482c20127 None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:12.135557 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0eaa2b5-8faa-4568-a92d-75c482c20127 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:12.135557 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0eaa2b5-8faa-4568-a92d-75c482c20127 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:12.136968 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0eaa2b5-8faa-4568-a92d-75c482c20127 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:12.142811 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21986ac0-73b2-477a-bdc1-76ec6aadb690 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:12.144591 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21986ac0-73b2-477a-bdc1-76ec6aadb690 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:12.156170 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 411/798] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jun 16 17:31:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:12.156170 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0eaa2b5-8faa-4568-a92d-75c482c20127 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:12.167680 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-21986ac0-73b2-477a-bdc1-76ec6aadb690 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 16 17:31:12.167680 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:12.167680 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:31:12.167680 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:31:12.167680 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:31:12.167680 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:31:12.167680 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:31:12.167680 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:31:12.167680 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:31:12.167680 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:31:12.167680 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:31:12.167680 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:31:12.167680 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:31:12.167680 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:31:12.167680 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:31:12.167680 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:31:12.168698 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:31:12.168698 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:31:12.168698 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:31:12.168698 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:31:12.168698 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:31:12.168698 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:31:12.168698 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:31:12.168698 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:31:12.168698 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:31:12.168698 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:31:12.168698 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:31:12.168698 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:31:12.168698 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:31:12.168698 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:31:12.168698 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:31:12.168698 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:31:12.168698 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:31:12.168698 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:31:12.169675 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:31:12.169675 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:31:12.169675 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:31:12.169675 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:31:12.169675 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:31:12.169675 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:31:12.169675 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:31:12.169675 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:31:12.169675 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 16 17:31:12.169675 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:31:12.174659 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 388/799] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Jun 16 17:31:11 2022] GET /identity/v3/users/swift => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:31:12.175646 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21986ac0-73b2-477a-bdc1-76ec6aadb690 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:12.281523 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21986ac0-73b2-477a-bdc1-76ec6aadb690 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:12.298616 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0eaa2b5-8faa-4568-a92d-75c482c20127 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:12.318879 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda3cf78-19e6-4c2f-9de6-0e37695c17be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:12.320000 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda3cf78-19e6-4c2f-9de6-0e37695c17be None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:12.320000 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda3cf78-19e6-4c2f-9de6-0e37695c17be None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:12.320257 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda3cf78-19e6-4c2f-9de6-0e37695c17be None admin] PATH_INFO: `/v3/projects/service` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:12.327837 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda3cf78-19e6-4c2f-9de6-0e37695c17be None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:12.329188 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda3cf78-19e6-4c2f-9de6-0e37695c17be None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:12.344248 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-fda3cf78-19e6-4c2f-9de6-0e37695c17be None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:31:12.344248 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:12.344248 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:31:12.344248 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:31:12.344248 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:31:12.344248 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:31:12.344248 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:31:12.344248 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:31:12.344248 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:31:12.344248 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:31:12.344248 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:31:12.344248 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:31:12.344248 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:31:12.344248 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:31:12.344248 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:31:12.345240 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:31:12.345240 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:31:12.345240 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:31:12.345240 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:31:12.345240 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:31:12.345240 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:31:12.345240 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:31:12.345240 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:31:12.345240 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:31:12.345240 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:31:12.345240 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:31:12.345240 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:31:12.345240 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:31:12.345240 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:31:12.345240 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:31:12.345240 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:31:12.345240 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:31:12.346198 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:31:12.346198 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:31:12.346198 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:31:12.346198 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:31:12.346198 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:31:12.346198 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:31:12.346198 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:31:12.346198 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:31:12.346198 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 389/800] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jun 16 17:31:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:31:12.346676 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda3cf78-19e6-4c2f-9de6-0e37695c17be None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:12.353269 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3961ba2e-c266-490f-834b-d47063e06eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:12.355523 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3961ba2e-c266-490f-834b-d47063e06eb9 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:12.356062 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3961ba2e-c266-490f-834b-d47063e06eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:12.356630 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3961ba2e-c266-490f-834b-d47063e06eb9 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:12.357546 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3961ba2e-c266-490f-834b-d47063e06eb9 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:12.360492 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3961ba2e-c266-490f-834b-d47063e06eb9 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:12.378961 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-3961ba2e-c266-490f-834b-d47063e06eb9 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 17:31:12.378961 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:12.378961 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 17:31:12.378961 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 17:31:12.378961 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 17:31:12.378961 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 17:31:12.378961 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 17:31:12.378961 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:31:12.378961 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 17:31:12.378961 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:31:12.378961 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:31:12.378961 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:31:12.378961 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:12.378961 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:31:12.378961 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:31:12.378961 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:31:12.379934 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:31:12.379934 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:31:12.379934 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:31:12.379934 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:31:12.379934 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:31:12.379934 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:31:12.379934 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:31:12.379934 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 17:31:12.379934 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 17:31:12.379934 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 17:31:12.379934 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 17:31:12.379934 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:31:12.379934 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:31:12.379934 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:31:12.379934 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:31:12.379934 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:31:12.380819 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:31:12.380819 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:31:12.380819 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:31:12.380819 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:31:12.380819 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:31:12.380819 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:31:12.380819 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:31:12.380819 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:31:12.380819 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:31:12.380819 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:31:12.380819 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:31:12.380819 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 17:31:12.380819 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:31:12.380819 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 17:31:12.380819 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:31:12.384545 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 412/801] 10.222.0.42 () {56 vars in 1129 bytes} [Thu Jun 16 17:31:12 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:31:12.385803 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3961ba2e-c266-490f-834b-d47063e06eb9 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:12.417781 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:12.468084 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:12.475682 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda3cf78-19e6-4c2f-9de6-0e37695c17be None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:12.514309 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075485df-34f2-4b58-85b7-85a0d2a2486e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:12.515017 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075485df-34f2-4b58-85b7-85a0d2a2486e None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:12.516114 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075485df-34f2-4b58-85b7-85a0d2a2486e None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:12.516412 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075485df-34f2-4b58-85b7-85a0d2a2486e None admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:12.517275 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075485df-34f2-4b58-85b7-85a0d2a2486e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:12.517585 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075485df-34f2-4b58-85b7-85a0d2a2486e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:12.519214 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075485df-34f2-4b58-85b7-85a0d2a2486e None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:12.533166 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3961ba2e-c266-490f-834b-d47063e06eb9 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:12.542798 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 390/802] 10.222.0.42 () {56 vars in 1135 bytes} [Thu Jun 16 17:31:12 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:12.543883 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075485df-34f2-4b58-85b7-85a0d2a2486e None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:12.582732 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e298063e-5669-4ad7-aeaa-dc27f0e0f393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:12.584738 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e298063e-5669-4ad7-aeaa-dc27f0e0f393 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:12.584950 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e298063e-5669-4ad7-aeaa-dc27f0e0f393 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:12.585179 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e298063e-5669-4ad7-aeaa-dc27f0e0f393 None admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:12.585976 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e298063e-5669-4ad7-aeaa-dc27f0e0f393 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:12.586284 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e298063e-5669-4ad7-aeaa-dc27f0e0f393 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:12.587726 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e298063e-5669-4ad7-aeaa-dc27f0e0f393 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:12.604430 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 413/803] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Jun 16 17:31:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:12.604430 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e298063e-5669-4ad7-aeaa-dc27f0e0f393 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:12.687833 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075485df-34f2-4b58-85b7-85a0d2a2486e None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:12.705527 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:12.730336 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee6c73c-8b0b-4c13-93d9-02aef0ec854d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:12.731332 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee6c73c-8b0b-4c13-93d9-02aef0ec854d None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:12.731854 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee6c73c-8b0b-4c13-93d9-02aef0ec854d None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:12.732311 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee6c73c-8b0b-4c13-93d9-02aef0ec854d None admin] PATH_INFO: `/v3/services` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:12.733236 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee6c73c-8b0b-4c13-93d9-02aef0ec854d None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:12.733728 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee6c73c-8b0b-4c13-93d9-02aef0ec854d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:12.734396 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e298063e-5669-4ad7-aeaa-dc27f0e0f393 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:12.735545 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee6c73c-8b0b-4c13-93d9-02aef0ec854d None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:12.741736 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 391/804] 10.222.0.42 () {56 vars in 1124 bytes} [Thu Jun 16 17:31:12 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:12.742798 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee6c73c-8b0b-4c13-93d9-02aef0ec854d None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:12.752552 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:12.778247 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ff3f88-1edc-4aca-b70b-b78b433922df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:12.779029 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ff3f88-1edc-4aca-b70b-b78b433922df None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:12.779317 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ff3f88-1edc-4aca-b70b-b78b433922df None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:12.779578 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ff3f88-1edc-4aca-b70b-b78b433922df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:12.791001 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ff3f88-1edc-4aca-b70b-b78b433922df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:12.792571 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ff3f88-1edc-4aca-b70b-b78b433922df None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:12.813248 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-32ff3f88-1edc-4aca-b70b-b78b433922df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:31:12.813248 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:12.813248 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:31:12.813248 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:31:12.813248 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:31:12.813248 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:31:12.813248 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:31:12.813248 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:31:12.813248 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:31:12.813248 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:31:12.813248 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:31:12.813248 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:31:12.813248 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:31:12.813248 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:31:12.813248 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:31:12.814403 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:31:12.814403 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:31:12.814403 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:31:12.814403 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:31:12.814403 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:31:12.814403 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:31:12.814403 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:31:12.814403 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:31:12.814403 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:31:12.814403 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:31:12.814403 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:31:12.814403 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:31:12.814403 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:31:12.814403 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:31:12.814403 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:31:12.814403 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:31:12.814403 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:31:12.815369 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:31:12.815369 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:31:12.815369 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:31:12.815369 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:31:12.821691 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 414/805] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jun 16 17:31:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:31:12.821691 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ff3f88-1edc-4aca-b70b-b78b433922df None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:12.865452 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee6c73c-8b0b-4c13-93d9-02aef0ec854d None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:12.910991 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96307b10-d5a0-428c-9838-5078dd30244d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:12.911925 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96307b10-d5a0-428c-9838-5078dd30244d None admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:12.912207 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96307b10-d5a0-428c-9838-5078dd30244d None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:12.912548 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96307b10-d5a0-428c-9838-5078dd30244d None admin] PATH_INFO: `/v3/projects/8abb87f2f141451a8c6ed832db82be18/users/71443c30b4554719b5549493bf3cc00c/roles/cd7dd29fcdc844c78544402dc30986fb` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:12.929054 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96307b10-d5a0-428c-9838-5078dd30244d None admin] RBAC: Authorizing `identity:create_grant(project_id=8abb87f2f141451a8c6ed832db82be18, user_id=71443c30b4554719b5549493bf3cc00c, role_id=cd7dd29fcdc844c78544402dc30986fb)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:12.930502 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96307b10-d5a0-428c-9838-5078dd30244d None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:12.945156 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 392/806] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Jun 16 17:31:12 2022] PUT /identity/v3/projects/8abb87f2f141451a8c6ed832db82be18/users/71443c30b4554719b5549493bf3cc00c/roles/cd7dd29fcdc844c78544402dc30986fb => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:31:12.947476 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ff3f88-1edc-4aca-b70b-b78b433922df None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:12.954804 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a994682-d6cb-47bb-8baa-05d16c9ad28e None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:12.955368 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a994682-d6cb-47bb-8baa-05d16c9ad28e None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:12.955808 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a994682-d6cb-47bb-8baa-05d16c9ad28e None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:12.962292 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 393/807] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:31:12 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:31:12.962292 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a994682-d6cb-47bb-8baa-05d16c9ad28e None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:12.997882 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:13.006916 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5719412f-d04a-4efc-abfe-b77baa6dd3fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:13.009028 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5719412f-d04a-4efc-abfe-b77baa6dd3fd None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:13.009649 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5719412f-d04a-4efc-abfe-b77baa6dd3fd None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:13.010084 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5719412f-d04a-4efc-abfe-b77baa6dd3fd None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:13.011200 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5719412f-d04a-4efc-abfe-b77baa6dd3fd None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:13.011760 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5719412f-d04a-4efc-abfe-b77baa6dd3fd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:13.013724 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5719412f-d04a-4efc-abfe-b77baa6dd3fd None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:13.022680 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 415/808] 10.222.0.42 () {56 vars in 1124 bytes} [Thu Jun 16 17:31:12 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:13.024332 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2957f2-42af-4260-a588-9a485da64d7c None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:13.024816 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2957f2-42af-4260-a588-9a485da64d7c None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:13.025837 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2957f2-42af-4260-a588-9a485da64d7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:13.050406 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-6c2957f2-42af-4260-a588-9a485da64d7c None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:31:13.052694 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-6c2957f2-42af-4260-a588-9a485da64d7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:31:13.087612 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:13.100113 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a994682-d6cb-47bb-8baa-05d16c9ad28e None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:13.146553 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795aea6d-3f0a-4123-a42d-3479b98b2109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:13.147633 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795aea6d-3f0a-4123-a42d-3479b98b2109 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:13.147927 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795aea6d-3f0a-4123-a42d-3479b98b2109 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:13.148211 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795aea6d-3f0a-4123-a42d-3479b98b2109 None admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:13.148981 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795aea6d-3f0a-4123-a42d-3479b98b2109 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:13.149349 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795aea6d-3f0a-4123-a42d-3479b98b2109 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:13.150785 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 416/809] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:31:13 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:31:13.150906 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795aea6d-3f0a-4123-a42d-3479b98b2109 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:13.151440 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2957f2-42af-4260-a588-9a485da64d7c None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:13.166732 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 394/810] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jun 16 17:31:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:13.172246 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795aea6d-3f0a-4123-a42d-3479b98b2109 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:13.271914 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:13.286535 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2957f2-42af-4260-a588-9a485da64d7c None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:13.292813 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795aea6d-3f0a-4123-a42d-3479b98b2109 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:13.329516 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c434aa7-da7c-4ca4-bc4b-ffb624717961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:13.330580 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0ae7f1-e29f-428b-a812-7f6bec1e125c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:13.330943 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c434aa7-da7c-4ca4-bc4b-ffb624717961 None admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:13.331376 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c434aa7-da7c-4ca4-bc4b-ffb624717961 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:13.331497 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0ae7f1-e29f-428b-a812-7f6bec1e125c None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:13.331715 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0ae7f1-e29f-428b-a812-7f6bec1e125c None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:13.331993 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0ae7f1-e29f-428b-a812-7f6bec1e125c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:13.332203 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c434aa7-da7c-4ca4-bc4b-ffb624717961 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:13.333115 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c434aa7-da7c-4ca4-bc4b-ffb624717961 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:13.335354 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c434aa7-da7c-4ca4-bc4b-ffb624717961 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:13.340124 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0ae7f1-e29f-428b-a812-7f6bec1e125c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:13.341824 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0ae7f1-e29f-428b-a812-7f6bec1e125c None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:13.361997 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-2d0ae7f1-e29f-428b-a812-7f6bec1e125c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:31:13.361997 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:13.361997 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:31:13.361997 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:31:13.361997 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:31:13.361997 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:31:13.361997 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:31:13.361997 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:31:13.361997 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:31:13.361997 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:31:13.361997 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:31:13.361997 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:31:13.361997 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:31:13.361997 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:31:13.361997 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:31:13.363133 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:31:13.363133 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:31:13.363133 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:31:13.363133 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:31:13.363133 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:31:13.363133 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:31:13.363133 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:31:13.363133 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:31:13.363133 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:31:13.363133 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:31:13.363133 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:31:13.363133 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:31:13.363133 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:31:13.363133 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:31:13.363133 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:31:13.363133 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:31:13.363133 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:31:13.364238 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:31:13.364238 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:31:13.364238 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:31:13.364238 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:31:13.364238 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:31:13.364238 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:31:13.364238 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:31:13.364238 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:31:13.365093 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 417/812] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Jun 16 17:31:13 2022] POST /identity/v3/endpoints => generated 346 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:31:13.366168 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c434aa7-da7c-4ca4-bc4b-ffb624717961 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:13.368172 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 395/812] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jun 16 17:31:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:31:13.369276 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0ae7f1-e29f-428b-a812-7f6bec1e125c None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:13.400829 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:13.580616 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0ae7f1-e29f-428b-a812-7f6bec1e125c None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:13.602793 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c434aa7-da7c-4ca4-bc4b-ffb624717961 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:13.636165 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577244b5-88bd-445e-a18f-b63602018189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:13.636970 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577244b5-88bd-445e-a18f-b63602018189 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:13.637266 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577244b5-88bd-445e-a18f-b63602018189 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:13.637548 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577244b5-88bd-445e-a18f-b63602018189 None admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:13.639439 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577244b5-88bd-445e-a18f-b63602018189 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:13.640577 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577244b5-88bd-445e-a18f-b63602018189 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:13.643023 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577244b5-88bd-445e-a18f-b63602018189 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:13.646849 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e95839d-2526-42da-b4df-d6d85eb344b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:13.648179 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e95839d-2526-42da-b4df-d6d85eb344b0 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:13.648497 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e95839d-2526-42da-b4df-d6d85eb344b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:13.648777 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e95839d-2526-42da-b4df-d6d85eb344b0 None admin] PATH_INFO: `/v3/services/network` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:13.649439 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e95839d-2526-42da-b4df-d6d85eb344b0 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:13.650816 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e95839d-2526-42da-b4df-d6d85eb344b0 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:13.660576 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 396/813] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Jun 16 17:31:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:13.669711 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-8e95839d-2526-42da-b4df-d6d85eb344b0 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 17:31:13.669711 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:13.669711 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 17:31:13.669711 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 17:31:13.669711 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 17:31:13.669711 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 17:31:13.669711 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 17:31:13.669711 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:31:13.669711 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 17:31:13.669711 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:31:13.669711 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:31:13.669711 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:31:13.669711 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:13.669711 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:31:13.669711 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:31:13.669711 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:31:13.669711 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:31:13.670780 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:31:13.670780 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:31:13.670780 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:31:13.670780 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:31:13.670780 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:31:13.670780 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:31:13.670780 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 17:31:13.670780 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 17:31:13.670780 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 17:31:13.670780 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 17:31:13.670780 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:31:13.670780 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:31:13.670780 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:31:13.670780 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:31:13.670780 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:31:13.670780 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:31:13.671669 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:31:13.671669 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:31:13.671669 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:31:13.671669 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:31:13.671669 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:31:13.671669 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:31:13.671669 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:31:13.671669 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:31:13.671669 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:31:13.671669 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:31:13.671669 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 17:31:13.671669 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:31:13.671669 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 17:31:13.671669 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:31:13.671669 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 418/814] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jun 16 17:31:13 2022] GET /identity/v3/services/network => generated 88 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:31:13.673846 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577244b5-88bd-445e-a18f-b63602018189 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:13.678106 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e95839d-2526-42da-b4df-d6d85eb344b0 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:13.687114 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:13.794193 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577244b5-88bd-445e-a18f-b63602018189 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:13.797369 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:13.809021 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e95839d-2526-42da-b4df-d6d85eb344b0 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:13.846934 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84076fa0-4b56-48f9-b610-e943439b5860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:13.847256 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7eea106-6cf6-42fc-87fe-1d5c57c8ee2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:13.847743 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84076fa0-4b56-48f9-b610-e943439b5860 None admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:13.847934 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eea106-6cf6-42fc-87fe-1d5c57c8ee2c None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:13.848054 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84076fa0-4b56-48f9-b610-e943439b5860 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:13.848241 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eea106-6cf6-42fc-87fe-1d5c57c8ee2c None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:13.848336 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84076fa0-4b56-48f9-b610-e943439b5860 None admin] PATH_INFO: `/v3/projects/8abb87f2f141451a8c6ed832db82be18/users/47b7a9cd615c41d6983a0007b475038c/roles/3ca2dcbbd29d4a31a3ab78f2299bc239` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:13.848519 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eea106-6cf6-42fc-87fe-1d5c57c8ee2c None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:13.849224 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7eea106-6cf6-42fc-87fe-1d5c57c8ee2c None admin] RBAC: Adding query filter params (name=network) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:13.849533 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7eea106-6cf6-42fc-87fe-1d5c57c8ee2c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:13.850328 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84076fa0-4b56-48f9-b610-e943439b5860 None admin] RBAC: Authorizing `identity:create_grant(project_id=8abb87f2f141451a8c6ed832db82be18, user_id=47b7a9cd615c41d6983a0007b475038c, role_id=3ca2dcbbd29d4a31a3ab78f2299bc239)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:13.850985 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7eea106-6cf6-42fc-87fe-1d5c57c8ee2c None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:13.851834 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84076fa0-4b56-48f9-b610-e943439b5860 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:13.856309 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 419/815] 10.222.0.42 () {56 vars in 1112 bytes} [Thu Jun 16 17:31:13 2022] GET /identity/v3/services?name=network => generated 124 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:13.857631 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0066d665-eabc-49fd-9d29-75826072cede None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:13.857923 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0066d665-eabc-49fd-9d29-75826072cede None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:13.858202 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0066d665-eabc-49fd-9d29-75826072cede None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:13.859870 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 420/816] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:31:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:31:13.864204 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0066d665-eabc-49fd-9d29-75826072cede None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:13.873365 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 397/817] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Jun 16 17:31:13 2022] PUT /identity/v3/projects/8abb87f2f141451a8c6ed832db82be18/users/47b7a9cd615c41d6983a0007b475038c/roles/3ca2dcbbd29d4a31a3ab78f2299bc239 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:31:13.874803 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2e8eac-cc66-402e-9309-281ffad9dbc5 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:13.875099 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2e8eac-cc66-402e-9309-281ffad9dbc5 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:13.875719 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2e8eac-cc66-402e-9309-281ffad9dbc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:13.897355 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-9a2e8eac-cc66-402e-9309-281ffad9dbc5 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:31:13.898703 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-9a2e8eac-cc66-402e-9309-281ffad9dbc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:31:13.987973 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:14.008551 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 398/818] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:31:13 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:31:14.028270 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2e8eac-cc66-402e-9309-281ffad9dbc5 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:14.063238 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0066d665-eabc-49fd-9d29-75826072cede None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:14.101784 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efe61a9-c1a6-48cb-a8fc-6b63fa1f05a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:14.102385 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efe61a9-c1a6-48cb-a8fc-6b63fa1f05a5 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:14.102670 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efe61a9-c1a6-48cb-a8fc-6b63fa1f05a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:14.102954 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efe61a9-c1a6-48cb-a8fc-6b63fa1f05a5 None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:14.103670 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efe61a9-c1a6-48cb-a8fc-6b63fa1f05a5 None admin] RBAC: Adding query filter params (type=network) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:14.103982 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efe61a9-c1a6-48cb-a8fc-6b63fa1f05a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:14.105413 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efe61a9-c1a6-48cb-a8fc-6b63fa1f05a5 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:14.111935 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 421/819] 10.222.0.42 () {56 vars in 1112 bytes} [Thu Jun 16 17:31:13 2022] GET /identity/v3/services?type=network => generated 350 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:14.117540 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efe61a9-c1a6-48cb-a8fc-6b63fa1f05a5 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:14.120605 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:14.149808 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2e8eac-cc66-402e-9309-281ffad9dbc5 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:14.193710 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6e8d33-2aeb-4b97-b038-a5be6e1f9aff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:14.195742 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6e8d33-2aeb-4b97-b038-a5be6e1f9aff None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:14.196364 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6e8d33-2aeb-4b97-b038-a5be6e1f9aff None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:14.196913 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6e8d33-2aeb-4b97-b038-a5be6e1f9aff None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:14.211976 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6e8d33-2aeb-4b97-b038-a5be6e1f9aff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:31:14.221251 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-8f6e8d33-2aeb-4b97-b038-a5be6e1f9aff None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:31:14.221251 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:14.221251 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:31:14.221251 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:31:14.221251 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:31:14.221251 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:31:14.221251 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:31:14.221251 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:31:14.221251 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:31:14.221251 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:31:14.221251 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:31:14.221251 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:31:14.221251 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:31:14.221251 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:31:14.221251 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:31:14.221251 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:31:14.222335 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:31:14.222335 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:31:14.222335 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:31:14.222335 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:31:14.222335 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:31:14.222335 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:31:14.222335 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:31:14.222335 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:31:14.222335 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:31:14.222335 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:31:14.222335 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:31:14.222335 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:31:14.222335 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:31:14.222335 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:31:14.222335 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:31:14.222335 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:31:14.223202 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:31:14.223202 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:31:14.223202 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:31:14.223202 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:31:14.223202 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:31:14.223202 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:31:14.223202 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:31:14.223202 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:31:14.223202 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:31:14.223202 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:31:14.223202 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:31:14.223202 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:31:14.230739 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 399/820] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Jun 16 17:31:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:31:14.231126 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efe61a9-c1a6-48cb-a8fc-6b63fa1f05a5 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:14.232675 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6e8d33-2aeb-4b97-b038-a5be6e1f9aff None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:14.293613 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c699c0b2-8ef8-48b7-aa54-bb6bdc8e765a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:14.294699 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c699c0b2-8ef8-48b7-aa54-bb6bdc8e765a None admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:14.294699 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c699c0b2-8ef8-48b7-aa54-bb6bdc8e765a None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:14.295031 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c699c0b2-8ef8-48b7-aa54-bb6bdc8e765a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:14.295670 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c699c0b2-8ef8-48b7-aa54-bb6bdc8e765a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:14.297131 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c699c0b2-8ef8-48b7-aa54-bb6bdc8e765a None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:14.308309 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:14.337780 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 422/821] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Jun 16 17:31:14 2022] POST /identity/v3/endpoints => generated 327 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:31:14.369425 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6e8d33-2aeb-4b97-b038-a5be6e1f9aff None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:14.420563 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2521004d-bde1-4f7d-aea9-68371f4edd8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:14.421832 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2521004d-bde1-4f7d-aea9-68371f4edd8f None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:14.422217 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2521004d-bde1-4f7d-aea9-68371f4edd8f None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:14.422497 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2521004d-bde1-4f7d-aea9-68371f4edd8f None admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:14.423266 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2521004d-bde1-4f7d-aea9-68371f4edd8f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:14.423576 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2521004d-bde1-4f7d-aea9-68371f4edd8f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:14.425227 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2521004d-bde1-4f7d-aea9-68371f4edd8f None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:14.435867 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 400/822] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jun 16 17:31:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:31:14.442161 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c699c0b2-8ef8-48b7-aa54-bb6bdc8e765a None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:14.557173 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:14.612919 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c699c0b2-8ef8-48b7-aa54-bb6bdc8e765a None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:14.649399 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb6b0a3-4b46-4942-a8c8-39bcb6ed7cb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:14.650540 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb6b0a3-4b46-4942-a8c8-39bcb6ed7cb9 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:14.651031 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb6b0a3-4b46-4942-a8c8-39bcb6ed7cb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:14.651520 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb6b0a3-4b46-4942-a8c8-39bcb6ed7cb9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:14.663511 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb6b0a3-4b46-4942-a8c8-39bcb6ed7cb9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:14.665284 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb6b0a3-4b46-4942-a8c8-39bcb6ed7cb9 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:14.683220 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-5bb6b0a3-4b46-4942-a8c8-39bcb6ed7cb9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:31:14.683220 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:14.683220 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:31:14.683220 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:31:14.683220 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:31:14.683220 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:31:14.683220 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:31:14.683220 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:31:14.683220 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:31:14.683220 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:31:14.683220 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:31:14.683220 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:31:14.683220 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:31:14.683220 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:31:14.683220 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:31:14.684158 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:31:14.684158 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:31:14.684158 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:31:14.684158 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:31:14.684158 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:31:14.684158 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:31:14.684158 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:31:14.684158 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:31:14.684158 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:31:14.684158 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:31:14.684158 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:31:14.684158 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:31:14.684158 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:31:14.684158 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:31:14.684158 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:31:14.684158 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:31:14.684158 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:31:14.685083 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:31:14.685083 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:31:14.685083 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:31:14.685083 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:31:14.691987 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 423/823] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jun 16 17:31:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:31:14.693658 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2521004d-bde1-4f7d-aea9-68371f4edd8f None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:14.801879 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2521004d-bde1-4f7d-aea9-68371f4edd8f None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:14.818093 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:14.841512 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce22f74-67bd-4798-af9c-d405ad7737e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:14.842208 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce22f74-67bd-4798-af9c-d405ad7737e5 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:14.842494 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce22f74-67bd-4798-af9c-d405ad7737e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:14.842772 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce22f74-67bd-4798-af9c-d405ad7737e5 None admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:14.844292 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce22f74-67bd-4798-af9c-d405ad7737e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:14.844694 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce22f74-67bd-4798-af9c-d405ad7737e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:14.846088 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce22f74-67bd-4798-af9c-d405ad7737e5 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:14.864269 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 401/824] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jun 16 17:31:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:14.865665 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb6b0a3-4b46-4942-a8c8-39bcb6ed7cb9 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:14.994503 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb6b0a3-4b46-4942-a8c8-39bcb6ed7cb9 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:15.006116 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:15.033154 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7ebbca-6f2e-4e7b-b503-518ad46aeca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:15.034284 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7ebbca-6f2e-4e7b-b503-518ad46aeca0 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:15.034761 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7ebbca-6f2e-4e7b-b503-518ad46aeca0 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:15.035216 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7ebbca-6f2e-4e7b-b503-518ad46aeca0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:15.043454 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7ebbca-6f2e-4e7b-b503-518ad46aeca0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:15.045181 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7ebbca-6f2e-4e7b-b503-518ad46aeca0 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:15.060923 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-bc7ebbca-6f2e-4e7b-b503-518ad46aeca0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 17:31:15.060923 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:15.060923 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:31:15.060923 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:31:15.060923 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:31:15.060923 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:31:15.060923 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:31:15.060923 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:31:15.060923 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:31:15.060923 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:31:15.060923 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:31:15.060923 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:31:15.060923 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:31:15.060923 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:31:15.060923 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:31:15.060923 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:31:15.062461 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:31:15.062461 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:31:15.062461 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:31:15.062461 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:31:15.062461 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:31:15.062461 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:31:15.062461 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:31:15.062461 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:31:15.062461 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:31:15.062461 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:31:15.062461 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:31:15.062461 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:31:15.062461 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:31:15.062461 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:31:15.062461 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:31:15.062461 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:31:15.062461 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:31:15.062461 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:31:15.063580 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:31:15.063580 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:31:15.063580 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:31:15.063580 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:31:15.063580 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:31:15.063580 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:31:15.063580 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:31:15.063580 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:31:15.063580 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 17:31:15.063580 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:31:15.070278 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 424/825] 10.222.0.42 () {56 vars in 1093 bytes} [Thu Jun 16 17:31:14 2022] GET /identity/v3/users/nova => generated 82 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:31:15.071839 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce22f74-67bd-4798-af9c-d405ad7737e5 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:15.178518 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce22f74-67bd-4798-af9c-d405ad7737e5 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:15.215377 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84a33ec-19a9-4301-83de-7318647f84ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:15.216305 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84a33ec-19a9-4301-83de-7318647f84ef None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:15.216843 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84a33ec-19a9-4301-83de-7318647f84ef None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:15.217287 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84a33ec-19a9-4301-83de-7318647f84ef None admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:15.218436 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84a33ec-19a9-4301-83de-7318647f84ef None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:15.218978 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84a33ec-19a9-4301-83de-7318647f84ef None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:15.220630 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84a33ec-19a9-4301-83de-7318647f84ef None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:15.243867 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 402/826] 10.222.0.42 () {56 vars in 1133 bytes} [Thu Jun 16 17:31:15 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:15.245571 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7ebbca-6f2e-4e7b-b503-518ad46aeca0 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:15.353701 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7ebbca-6f2e-4e7b-b503-518ad46aeca0 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:15.393939 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af7a751-eaf7-47a2-9ef2-601b9097977d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:15.394624 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af7a751-eaf7-47a2-9ef2-601b9097977d None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:15.394941 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af7a751-eaf7-47a2-9ef2-601b9097977d None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:15.395259 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af7a751-eaf7-47a2-9ef2-601b9097977d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:15.445846 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:15.460506 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d9c5cf-b0ba-4697-9150-1402c55ad379 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:15.460790 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d9c5cf-b0ba-4697-9150-1402c55ad379 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:15.461109 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d9c5cf-b0ba-4697-9150-1402c55ad379 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:15.462562 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 403/827] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:31:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:31:15.469565 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c252b5ec-e6b1-40ba-b20e-1576503eb8ea None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:15.469913 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c252b5ec-e6b1-40ba-b20e-1576503eb8ea None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:15.470198 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c252b5ec-e6b1-40ba-b20e-1576503eb8ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:15.480384 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af7a751-eaf7-47a2-9ef2-601b9097977d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:15.482242 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af7a751-eaf7-47a2-9ef2-601b9097977d None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:15.489594 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-c252b5ec-e6b1-40ba-b20e-1576503eb8ea None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:31:15.490801 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-c252b5ec-e6b1-40ba-b20e-1576503eb8ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:31:15.519956 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:15.584404 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 404/828] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:31:15 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 17:31:15.603122 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c252b5ec-e6b1-40ba-b20e-1576503eb8ea None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:15.607620 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 425/829] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jun 16 17:31:15 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 17:31:15.617346 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af7a751-eaf7-47a2-9ef2-601b9097977d None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:15.696785 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:15.724448 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c252b5ec-e6b1-40ba-b20e-1576503eb8ea None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:15.729420 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af7a751-eaf7-47a2-9ef2-601b9097977d None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:15.764356 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8cb17d-684e-4988-9df5-f55fd341a825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:15.766394 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8cb17d-684e-4988-9df5-f55fd341a825 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:15.766508 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2ca258-7352-4d23-b935-f36f7db267d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:15.767022 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8cb17d-684e-4988-9df5-f55fd341a825 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:15.767355 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2ca258-7352-4d23-b935-f36f7db267d7 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:15.767642 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2ca258-7352-4d23-b935-f36f7db267d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:15.767922 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2ca258-7352-4d23-b935-f36f7db267d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:15.768098 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8cb17d-684e-4988-9df5-f55fd341a825 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:15.778798 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2ca258-7352-4d23-b935-f36f7db267d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:15.780190 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2ca258-7352-4d23-b935-f36f7db267d7 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:15.780314 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8cb17d-684e-4988-9df5-f55fd341a825 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:31:15.788866 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-6d8cb17d-684e-4988-9df5-f55fd341a825 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:31:15.788866 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:15.788866 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:31:15.788866 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:31:15.788866 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:31:15.788866 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:31:15.788866 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:31:15.788866 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:31:15.788866 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:31:15.788866 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:31:15.788866 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:31:15.788866 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:31:15.788866 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:31:15.788866 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:31:15.788866 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:31:15.788866 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:31:15.789809 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:31:15.789809 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:31:15.789809 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:31:15.789809 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:31:15.789809 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:31:15.789809 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:31:15.789809 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:31:15.789809 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:31:15.789809 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:31:15.789809 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:31:15.789809 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:31:15.789809 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:31:15.789809 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:31:15.789809 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:31:15.789809 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:31:15.789809 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:31:15.790645 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:31:15.790645 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:31:15.790645 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:31:15.790645 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:31:15.790645 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:31:15.790645 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:31:15.790645 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:31:15.790645 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:31:15.790645 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:31:15.790645 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:31:15.790645 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:31:15.790645 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:31:15.798218 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 405/830] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Jun 16 17:31:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:31:15.799101 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-bc2ca258-7352-4d23-b935-f36f7db267d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:31:15.799101 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:15.799101 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:31:15.799101 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:31:15.799101 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:31:15.799101 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:31:15.799101 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:31:15.799101 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:31:15.799101 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:31:15.799101 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:31:15.799101 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:31:15.799101 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:31:15.799101 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:31:15.799101 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:31:15.799101 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:31:15.800047 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:31:15.800047 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:31:15.800047 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:31:15.800047 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:31:15.800047 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:31:15.800047 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:31:15.800047 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:31:15.800047 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:31:15.800047 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:31:15.800047 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:31:15.800047 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:31:15.800047 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:31:15.800047 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:31:15.800047 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:31:15.800047 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:31:15.800047 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:31:15.800047 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:31:15.800989 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:31:15.800989 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:31:15.800989 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:31:15.800989 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:31:15.801855 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8cb17d-684e-4988-9df5-f55fd341a825 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:15.804452 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 426/831] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jun 16 17:31:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:31:15.805900 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2ca258-7352-4d23-b935-f36f7db267d7 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:15.837500 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:15.933295 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2ca258-7352-4d23-b935-f36f7db267d7 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:15.940294 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8cb17d-684e-4988-9df5-f55fd341a825 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:15.959063 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:15.973233 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2070ef-b3de-4631-b499-0ed63353f9ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:15.974193 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2070ef-b3de-4631-b499-0ed63353f9ce None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:15.974669 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2070ef-b3de-4631-b499-0ed63353f9ce None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:15.974953 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2070ef-b3de-4631-b499-0ed63353f9ce None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:15.975674 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2070ef-b3de-4631-b499-0ed63353f9ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:15.975982 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2070ef-b3de-4631-b499-0ed63353f9ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:15.977480 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2070ef-b3de-4631-b499-0ed63353f9ce None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:15.979218 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70183ca7-d909-40ad-981e-7a8467349adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:15.979882 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70183ca7-d909-40ad-981e-7a8467349adf None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:15.980162 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70183ca7-d909-40ad-981e-7a8467349adf None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:15.980465 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70183ca7-d909-40ad-981e-7a8467349adf None admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:15.981209 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70183ca7-d909-40ad-981e-7a8467349adf None admin] RBAC: Adding query filter params (name=service) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:15.981510 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70183ca7-d909-40ad-981e-7a8467349adf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:15.982868 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70183ca7-d909-40ad-981e-7a8467349adf None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:15.994961 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 427/832] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jun 16 17:31:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:15.995149 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 406/833] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Jun 16 17:31:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:15.995244 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba43df4-08c1-4991-b6cf-3071a3c08f57 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:15.996536 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba43df4-08c1-4991-b6cf-3071a3c08f57 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:15.996826 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba43df4-08c1-4991-b6cf-3071a3c08f57 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:15.997128 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70183ca7-d909-40ad-981e-7a8467349adf None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:16.002203 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 428/834] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:31:15 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:31:16.002203 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba43df4-08c1-4991-b6cf-3071a3c08f57 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:16.088934 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:16.108225 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70183ca7-d909-40ad-981e-7a8467349adf None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:16.112651 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba43df4-08c1-4991-b6cf-3071a3c08f57 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:16.149486 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c936ae-c3c2-42c3-9635-373118ba527e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:16.150715 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c936ae-c3c2-42c3-9635-373118ba527e None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:16.150998 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c936ae-c3c2-42c3-9635-373118ba527e None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:16.151370 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c936ae-c3c2-42c3-9635-373118ba527e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:16.153456 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcf0671-0f44-47f7-8d1c-204973886777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:16.154623 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcf0671-0f44-47f7-8d1c-204973886777 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:16.154905 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcf0671-0f44-47f7-8d1c-204973886777 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:16.155231 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcf0671-0f44-47f7-8d1c-204973886777 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:16.159227 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c936ae-c3c2-42c3-9635-373118ba527e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:16.160701 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c936ae-c3c2-42c3-9635-373118ba527e None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:16.168524 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcf0671-0f44-47f7-8d1c-204973886777 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:16.169981 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcf0671-0f44-47f7-8d1c-204973886777 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:16.175252 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-66c936ae-c3c2-42c3-9635-373118ba527e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:31:16.175252 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:16.175252 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:31:16.175252 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:31:16.175252 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:31:16.175252 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:31:16.175252 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:31:16.175252 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:31:16.175252 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:31:16.175252 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:31:16.175252 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:31:16.175252 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:31:16.175252 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:31:16.175252 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:31:16.175252 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:31:16.176220 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:31:16.176220 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:31:16.176220 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:31:16.176220 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:31:16.176220 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:31:16.176220 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:31:16.176220 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:31:16.176220 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:31:16.176220 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:31:16.176220 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:31:16.176220 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:31:16.176220 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:31:16.176220 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:31:16.176220 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:31:16.176220 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:31:16.176220 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:31:16.176220 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:31:16.177141 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:31:16.177141 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:31:16.177141 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:31:16.177141 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:31:16.177141 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:31:16.177141 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:31:16.177141 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:31:16.177141 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:31:16.180530 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 407/835] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jun 16 17:31:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:31:16.182126 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf692d5c-814a-49c1-84be-2cef874cabc1 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:16.182418 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf692d5c-814a-49c1-84be-2cef874cabc1 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:16.182699 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf692d5c-814a-49c1-84be-2cef874cabc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:16.189890 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-1bcf0671-0f44-47f7-8d1c-204973886777 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:31:16.189890 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:16.189890 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:31:16.189890 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:31:16.189890 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:31:16.189890 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:31:16.189890 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:31:16.189890 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:31:16.189890 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:31:16.189890 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:31:16.189890 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:31:16.189890 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:31:16.189890 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:31:16.189890 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:31:16.189890 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:31:16.190945 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:31:16.190945 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:31:16.190945 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:31:16.190945 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:31:16.190945 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:31:16.190945 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:31:16.190945 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:31:16.190945 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:31:16.190945 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:31:16.190945 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:31:16.190945 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:31:16.190945 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:31:16.190945 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:31:16.190945 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:31:16.190945 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:31:16.190945 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:31:16.190945 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:31:16.192540 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:31:16.192540 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:31:16.192540 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:31:16.192540 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:31:16.192540 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 429/836] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jun 16 17:31:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:31:16.192540 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcf0671-0f44-47f7-8d1c-204973886777 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:16.207214 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-bf692d5c-814a-49c1-84be-2cef874cabc1 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:31:16.208446 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-bf692d5c-814a-49c1-84be-2cef874cabc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:31:16.230456 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:16.299268 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 408/837] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:31:16 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:31:16.300543 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf692d5c-814a-49c1-84be-2cef874cabc1 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:16.330750 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcf0671-0f44-47f7-8d1c-204973886777 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:16.377437 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c005497c-b2f1-488f-9097-855a339dd561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:16.378053 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c005497c-b2f1-488f-9097-855a339dd561 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:16.378288 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c005497c-b2f1-488f-9097-855a339dd561 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:16.378557 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c005497c-b2f1-488f-9097-855a339dd561 None admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:16.379098 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:16.380142 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c005497c-b2f1-488f-9097-855a339dd561 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:16.380592 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c005497c-b2f1-488f-9097-855a339dd561 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:16.382355 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c005497c-b2f1-488f-9097-855a339dd561 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:16.399952 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 430/838] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Jun 16 17:31:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:16.401143 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c005497c-b2f1-488f-9097-855a339dd561 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:16.411985 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf692d5c-814a-49c1-84be-2cef874cabc1 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:16.466826 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc90d04-18b3-4218-8376-27b333c1dbb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:16.467854 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc90d04-18b3-4218-8376-27b333c1dbb7 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:16.468862 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc90d04-18b3-4218-8376-27b333c1dbb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:16.468996 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc90d04-18b3-4218-8376-27b333c1dbb7 None admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:16.469883 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc90d04-18b3-4218-8376-27b333c1dbb7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:16.470372 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc90d04-18b3-4218-8376-27b333c1dbb7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:16.472615 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc90d04-18b3-4218-8376-27b333c1dbb7 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:16.494843 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 409/839] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jun 16 17:31:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:16.494843 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc90d04-18b3-4218-8376-27b333c1dbb7 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:16.545044 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:16.554389 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c005497c-b2f1-488f-9097-855a339dd561 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:16.592061 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc56243c-30fe-480e-8f02-957a623ed27b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:16.592975 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc56243c-30fe-480e-8f02-957a623ed27b None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:16.593384 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc56243c-30fe-480e-8f02-957a623ed27b None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:16.593580 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc56243c-30fe-480e-8f02-957a623ed27b None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:16.594297 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc56243c-30fe-480e-8f02-957a623ed27b None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:16.595752 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc56243c-30fe-480e-8f02-957a623ed27b None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:16.606148 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc90d04-18b3-4218-8376-27b333c1dbb7 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:16.607516 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-cc56243c-30fe-480e-8f02-957a623ed27b None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 17:31:16.607516 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:16.607516 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 17:31:16.607516 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 17:31:16.607516 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 17:31:16.607516 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 17:31:16.607516 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 17:31:16.607516 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:31:16.607516 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 17:31:16.607516 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:31:16.607516 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:31:16.607516 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:31:16.607516 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:16.607516 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:31:16.607516 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:31:16.607516 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:31:16.607516 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:31:16.608640 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:31:16.608640 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:31:16.608640 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:31:16.608640 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:31:16.608640 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:31:16.608640 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:31:16.608640 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 17:31:16.608640 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 17:31:16.608640 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 17:31:16.608640 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 17:31:16.608640 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:31:16.608640 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:31:16.608640 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:31:16.608640 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:31:16.608640 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:31:16.608640 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:31:16.609550 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:31:16.609550 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:31:16.609550 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:31:16.609550 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:31:16.609550 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:31:16.609550 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:31:16.609550 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:31:16.609550 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:31:16.609550 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:31:16.609550 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:31:16.609550 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 17:31:16.609550 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:31:16.609550 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 17:31:16.609550 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:31:16.609550 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 431/840] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Jun 16 17:31:16 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:31:16.610572 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc56243c-30fe-480e-8f02-957a623ed27b None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:16.647173 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e4fb46-873f-4886-bfef-e68d22bde3e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:16.647743 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e4fb46-873f-4886-bfef-e68d22bde3e0 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:16.648022 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e4fb46-873f-4886-bfef-e68d22bde3e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:16.648293 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e4fb46-873f-4886-bfef-e68d22bde3e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:16.649146 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e4fb46-873f-4886-bfef-e68d22bde3e0 None admin] RBAC: Adding query filter params (role.id=3ca2dcbbd29d4a31a3ab78f2299bc239, scope.project.id=8abb87f2f141451a8c6ed832db82be18, user.id=f86c16179a70474a817be9fda33ece9d) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:16.649455 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e4fb46-873f-4886-bfef-e68d22bde3e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:16.650878 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e4fb46-873f-4886-bfef-e68d22bde3e0 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:16.652733 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:16.657652 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 410/841] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jun 16 17:31:16 2022] GET /identity/v3/role_assignments?user.id=f86c16179a70474a817be9fda33ece9d&scope.project.id=8abb87f2f141451a8c6ed832db82be18&role.id=3ca2dcbbd29d4a31a3ab78f2299bc239 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:16.661972 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78a0e2c-4a61-4b05-9016-c838c84a92e8 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:16.662575 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78a0e2c-4a61-4b05-9016-c838c84a92e8 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:16.662860 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78a0e2c-4a61-4b05-9016-c838c84a92e8 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:16.664463 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 411/842] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:31:16 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:31:16.665172 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78a0e2c-4a61-4b05-9016-c838c84a92e8 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:16.728993 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc56243c-30fe-480e-8f02-957a623ed27b None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:16.769077 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08e0186-7989-43d7-a5d4-7238708ce571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:16.769908 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08e0186-7989-43d7-a5d4-7238708ce571 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:16.770195 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08e0186-7989-43d7-a5d4-7238708ce571 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:16.770472 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08e0186-7989-43d7-a5d4-7238708ce571 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:16.778939 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08e0186-7989-43d7-a5d4-7238708ce571 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:16.780441 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08e0186-7989-43d7-a5d4-7238708ce571 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:16.797654 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-d08e0186-7989-43d7-a5d4-7238708ce571 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 17:31:16.797654 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:16.797654 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:31:16.797654 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:31:16.797654 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:31:16.797654 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:31:16.797654 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:31:16.797654 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:31:16.797654 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:31:16.797654 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:31:16.797654 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:31:16.797654 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:31:16.797654 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:31:16.797654 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:31:16.797654 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:31:16.797654 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:31:16.799025 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:31:16.799025 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:31:16.799025 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:31:16.799025 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:31:16.799025 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:31:16.799025 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:31:16.799025 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:31:16.799025 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:31:16.799025 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:31:16.799025 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:31:16.799025 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:31:16.799025 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:31:16.799025 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:31:16.799025 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:31:16.799025 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:31:16.799025 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:31:16.799025 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:31:16.799025 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:31:16.799981 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:31:16.799981 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:31:16.799981 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:31:16.799981 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:31:16.799981 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:31:16.799981 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:31:16.799981 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:31:16.799981 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:31:16.799981 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 17:31:16.799981 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:31:16.799981 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 432/843] 10.222.0.42 () {56 vars in 1117 bytes} [Thu Jun 16 17:31:16 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:31:16.799981 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2a5755-5aed-45b4-9ae0-1e9490bc0fd0 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:16.799981 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2a5755-5aed-45b4-9ae0-1e9490bc0fd0 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:16.800729 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2a5755-5aed-45b4-9ae0-1e9490bc0fd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:16.802768 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78a0e2c-4a61-4b05-9016-c838c84a92e8 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:16.819498 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-1c2a5755-5aed-45b4-9ae0-1e9490bc0fd0 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:31:16.820766 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-1c2a5755-5aed-45b4-9ae0-1e9490bc0fd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:31:16.838770 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:16.847749 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7498c3-e859-4146-b23d-5dbb68c4e574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:16.848255 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7498c3-e859-4146-b23d-5dbb68c4e574 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:16.848546 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7498c3-e859-4146-b23d-5dbb68c4e574 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:16.848703 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7498c3-e859-4146-b23d-5dbb68c4e574 None admin] PATH_INFO: `/v3/services` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:16.849411 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7498c3-e859-4146-b23d-5dbb68c4e574 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:16.849820 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7498c3-e859-4146-b23d-5dbb68c4e574 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:16.851093 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7498c3-e859-4146-b23d-5dbb68c4e574 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:16.855881 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 412/844] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Jun 16 17:31:16 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:16.856754 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7498c3-e859-4146-b23d-5dbb68c4e574 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:16.909858 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 433/845] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:31:16 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:31:16.910636 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2a5755-5aed-45b4-9ae0-1e9490bc0fd0 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:16.946074 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:16.993409 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7498c3-e859-4146-b23d-5dbb68c4e574 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:17.023716 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2a5755-5aed-45b4-9ae0-1e9490bc0fd0 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:17.033411 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f184b66e-d495-41cf-9b60-31bf8ffd2503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:17.034099 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f184b66e-d495-41cf-9b60-31bf8ffd2503 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:17.034387 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f184b66e-d495-41cf-9b60-31bf8ffd2503 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:17.034662 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f184b66e-d495-41cf-9b60-31bf8ffd2503 None admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:17.035508 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f184b66e-d495-41cf-9b60-31bf8ffd2503 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:17.035821 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f184b66e-d495-41cf-9b60-31bf8ffd2503 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:17.037289 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f184b66e-d495-41cf-9b60-31bf8ffd2503 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:17.061814 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 413/846] 10.222.0.42 () {56 vars in 1149 bytes} [Thu Jun 16 17:31:16 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:17.061814 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f184b66e-d495-41cf-9b60-31bf8ffd2503 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:17.065423 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e520d6fb-13b8-4cbb-830b-ff8b9f5b8caa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:17.066101 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e520d6fb-13b8-4cbb-830b-ff8b9f5b8caa None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:17.066386 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e520d6fb-13b8-4cbb-830b-ff8b9f5b8caa None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:17.066660 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e520d6fb-13b8-4cbb-830b-ff8b9f5b8caa None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:17.067373 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e520d6fb-13b8-4cbb-830b-ff8b9f5b8caa None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:17.067682 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e520d6fb-13b8-4cbb-830b-ff8b9f5b8caa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:17.069101 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e520d6fb-13b8-4cbb-830b-ff8b9f5b8caa None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:17.074506 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 434/847] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Jun 16 17:31:16 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:17.075383 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e520d6fb-13b8-4cbb-830b-ff8b9f5b8caa None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:17.094742 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:17.198045 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e520d6fb-13b8-4cbb-830b-ff8b9f5b8caa None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:17.225102 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f184b66e-d495-41cf-9b60-31bf8ffd2503 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:17.245229 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2a598e-448c-45cf-83c0-57fefffb2888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:17.246156 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2a598e-448c-45cf-83c0-57fefffb2888 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:17.246439 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2a598e-448c-45cf-83c0-57fefffb2888 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:17.246716 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2a598e-448c-45cf-83c0-57fefffb2888 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:17.248233 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:17.272891 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd20429-5a3a-4229-8ca3-658f22df75c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:17.273698 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd20429-5a3a-4229-8ca3-658f22df75c1 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:17.273987 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd20429-5a3a-4229-8ca3-658f22df75c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:17.274395 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd20429-5a3a-4229-8ca3-658f22df75c1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:17.288220 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd20429-5a3a-4229-8ca3-658f22df75c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:31:17.297897 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-5bd20429-5a3a-4229-8ca3-658f22df75c1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:31:17.297897 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:17.297897 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:31:17.297897 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:31:17.297897 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:31:17.297897 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:31:17.297897 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:31:17.297897 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:31:17.297897 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:31:17.297897 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:31:17.297897 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:31:17.297897 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:31:17.297897 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:31:17.297897 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:31:17.297897 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:31:17.297897 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:31:17.299030 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:31:17.299030 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:31:17.299030 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:31:17.299030 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:31:17.299030 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:31:17.299030 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:31:17.299030 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:31:17.299030 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:31:17.299030 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:31:17.299030 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:31:17.299030 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:31:17.299030 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:31:17.299030 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:31:17.299030 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:31:17.299030 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:31:17.299030 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:31:17.304911 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:31:17.304911 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:31:17.304911 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:31:17.304911 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:31:17.304911 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:31:17.304911 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:31:17.304911 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:31:17.304911 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:31:17.304911 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:31:17.304911 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:31:17.304911 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:31:17.304911 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:31:17.304911 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 414/848] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Jun 16 17:31:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:31:17.305840 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd20429-5a3a-4229-8ca3-658f22df75c1 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:17.335095 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2a598e-448c-45cf-83c0-57fefffb2888 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:17.337216 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2a598e-448c-45cf-83c0-57fefffb2888 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:17.354987 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:17.450395 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd20429-5a3a-4229-8ca3-658f22df75c1 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:17.474352 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 435/849] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jun 16 17:31:17 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 17:31:17.484545 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2a598e-448c-45cf-83c0-57fefffb2888 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:17.499402 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137ecef5-70d7-4d9c-bbed-775de5391b69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:17.500171 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137ecef5-70d7-4d9c-bbed-775de5391b69 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:17.500592 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137ecef5-70d7-4d9c-bbed-775de5391b69 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:17.500930 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137ecef5-70d7-4d9c-bbed-775de5391b69 None admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:17.501796 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137ecef5-70d7-4d9c-bbed-775de5391b69 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:17.502167 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137ecef5-70d7-4d9c-bbed-775de5391b69 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:17.503611 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137ecef5-70d7-4d9c-bbed-775de5391b69 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:17.512901 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 415/850] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jun 16 17:31:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:17.518889 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137ecef5-70d7-4d9c-bbed-775de5391b69 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:17.571062 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:17.610548 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2a598e-448c-45cf-83c0-57fefffb2888 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:17.625548 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:17.641662 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137ecef5-70d7-4d9c-bbed-775de5391b69 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:17.649582 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df173f8-6b86-4f20-9975-79a3e6e4023f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:17.650391 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df173f8-6b86-4f20-9975-79a3e6e4023f None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:17.650672 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df173f8-6b86-4f20-9975-79a3e6e4023f None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:17.650946 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df173f8-6b86-4f20-9975-79a3e6e4023f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:17.665525 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df173f8-6b86-4f20-9975-79a3e6e4023f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:17.666967 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df173f8-6b86-4f20-9975-79a3e6e4023f None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:17.684768 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce1eb42-ab5b-4235-978f-176d5cccd458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:17.685566 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce1eb42-ab5b-4235-978f-176d5cccd458 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:17.685849 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce1eb42-ab5b-4235-978f-176d5cccd458 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:17.685969 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-8df173f8-6b86-4f20-9975-79a3e6e4023f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:31:17.685969 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:17.685969 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:31:17.685969 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:31:17.685969 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:31:17.685969 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:31:17.685969 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:31:17.685969 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:31:17.685969 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:31:17.685969 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:31:17.685969 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:31:17.685969 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:31:17.685969 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:31:17.685969 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:31:17.685969 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:31:17.686776 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:31:17.686776 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:31:17.686776 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:31:17.686776 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:31:17.686776 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:31:17.686776 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:31:17.686776 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:31:17.686776 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:31:17.686776 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:31:17.686776 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:31:17.686776 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:31:17.686776 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:31:17.686776 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:31:17.686776 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:31:17.686776 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:31:17.686776 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:31:17.686776 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:31:17.687877 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:31:17.687877 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:31:17.687877 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:31:17.687877 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:31:17.688106 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce1eb42-ab5b-4235-978f-176d5cccd458 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:17.688175 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 436/851] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jun 16 17:31:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:31:17.693372 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df173f8-6b86-4f20-9975-79a3e6e4023f None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:17.706241 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce1eb42-ab5b-4235-978f-176d5cccd458 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:17.708358 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce1eb42-ab5b-4235-978f-176d5cccd458 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:17.739395 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-fce1eb42-ab5b-4235-978f-176d5cccd458 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:31:17.739395 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:17.739395 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:31:17.739395 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:31:17.739395 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:31:17.739395 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:31:17.739395 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:31:17.739395 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:31:17.739395 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:31:17.739395 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:31:17.739395 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:31:17.739395 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:31:17.739395 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:31:17.739395 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:31:17.739395 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:31:17.740333 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:31:17.740333 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:31:17.740333 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:31:17.740333 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:31:17.740333 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:31:17.740333 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:31:17.740333 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:31:17.740333 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:31:17.740333 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:31:17.740333 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:31:17.740333 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:31:17.740333 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:31:17.740333 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:31:17.740333 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:31:17.740333 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:31:17.740333 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:31:17.740333 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:31:17.741270 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:31:17.741270 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:31:17.741270 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:31:17.741270 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:31:17.744496 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 416/852] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jun 16 17:31:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:31:17.747219 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce1eb42-ab5b-4235-978f-176d5cccd458 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:17.814755 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df173f8-6b86-4f20-9975-79a3e6e4023f None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:17.857669 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a56afc9-591b-4029-9fc6-b81f3713fa0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:17.859108 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a56afc9-591b-4029-9fc6-b81f3713fa0a None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:17.859108 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a56afc9-591b-4029-9fc6-b81f3713fa0a None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:17.859108 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a56afc9-591b-4029-9fc6-b81f3713fa0a None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:17.860350 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a56afc9-591b-4029-9fc6-b81f3713fa0a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:17.860704 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a56afc9-591b-4029-9fc6-b81f3713fa0a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:17.862120 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a56afc9-591b-4029-9fc6-b81f3713fa0a None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:17.879983 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 437/853] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jun 16 17:31:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:17.885654 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a56afc9-591b-4029-9fc6-b81f3713fa0a None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:17.894727 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:17.903357 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce1eb42-ab5b-4235-978f-176d5cccd458 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:17.915646 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:17.944903 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad512d37-af2b-46b1-abbc-3dc2970c9108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:17.945962 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad512d37-af2b-46b1-abbc-3dc2970c9108 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:17.946534 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad512d37-af2b-46b1-abbc-3dc2970c9108 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:17.947016 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad512d37-af2b-46b1-abbc-3dc2970c9108 None admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:17.948001 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad512d37-af2b-46b1-abbc-3dc2970c9108 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:17.948540 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad512d37-af2b-46b1-abbc-3dc2970c9108 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:17.950240 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad512d37-af2b-46b1-abbc-3dc2970c9108 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:17.968034 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 417/854] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jun 16 17:31:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:17.970086 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad512d37-af2b-46b1-abbc-3dc2970c9108 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:18.012781 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a56afc9-591b-4029-9fc6-b81f3713fa0a None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:18.060543 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15f14f9-3c07-4d04-8560-a613d4c44c0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:18.061395 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15f14f9-3c07-4d04-8560-a613d4c44c0a None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:18.061683 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15f14f9-3c07-4d04-8560-a613d4c44c0a None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:18.062594 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15f14f9-3c07-4d04-8560-a613d4c44c0a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:18.074206 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15f14f9-3c07-4d04-8560-a613d4c44c0a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:18.075921 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15f14f9-3c07-4d04-8560-a613d4c44c0a None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:18.077217 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad512d37-af2b-46b1-abbc-3dc2970c9108 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:18.095265 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-a15f14f9-3c07-4d04-8560-a613d4c44c0a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:31:18.095265 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:18.095265 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:31:18.095265 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:31:18.095265 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:31:18.095265 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:31:18.095265 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:31:18.095265 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:31:18.095265 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:31:18.095265 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:31:18.095265 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:31:18.095265 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:31:18.095265 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:31:18.095265 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:31:18.095265 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:31:18.096210 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:31:18.096210 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:31:18.096210 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:31:18.096210 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:31:18.096210 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:31:18.096210 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:31:18.096210 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:31:18.096210 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:31:18.096210 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:31:18.096210 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:31:18.096210 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:31:18.096210 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:31:18.096210 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:31:18.096210 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:31:18.096210 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:31:18.096210 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:31:18.096210 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:31:18.097145 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:31:18.097145 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:31:18.097145 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:31:18.097145 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:31:18.097145 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:31:18.097145 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:31:18.097145 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:31:18.097145 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:31:18.097145 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 438/855] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jun 16 17:31:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:31:18.102191 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15f14f9-3c07-4d04-8560-a613d4c44c0a None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:18.127926 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b15d090-79c7-4608-a7b3-4d6785d1bc5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:18.129100 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b15d090-79c7-4608-a7b3-4d6785d1bc5a None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:18.129713 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b15d090-79c7-4608-a7b3-4d6785d1bc5a None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:18.130187 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b15d090-79c7-4608-a7b3-4d6785d1bc5a None admin] PATH_INFO: `/v3/users/swift` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:18.138703 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b15d090-79c7-4608-a7b3-4d6785d1bc5a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:18.141394 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b15d090-79c7-4608-a7b3-4d6785d1bc5a None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:18.158473 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-2b15d090-79c7-4608-a7b3-4d6785d1bc5a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 16 17:31:18.158473 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:18.158473 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:31:18.158473 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:31:18.158473 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:31:18.158473 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:31:18.158473 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:31:18.158473 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:31:18.158473 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:31:18.158473 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:31:18.158473 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:31:18.158473 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:31:18.158473 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:31:18.158473 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:31:18.158473 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:31:18.158473 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:31:18.159472 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:31:18.159472 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:31:18.159472 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:31:18.159472 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:31:18.159472 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:31:18.159472 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:31:18.159472 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:31:18.159472 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:31:18.159472 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:31:18.159472 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:31:18.159472 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:31:18.159472 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:31:18.159472 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:31:18.159472 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:31:18.159472 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:31:18.159472 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:31:18.159472 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:31:18.159472 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:31:18.160430 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:31:18.160430 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:31:18.160430 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:31:18.160430 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:31:18.160430 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:31:18.160430 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:31:18.160430 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:31:18.160430 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:31:18.160430 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 16 17:31:18.160430 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:31:18.163174 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 418/856] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Jun 16 17:31:17 2022] GET /identity/v3/users/swift => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:31:18.170144 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b15d090-79c7-4608-a7b3-4d6785d1bc5a None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:18.189485 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:18.199700 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:18.217244 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15f14f9-3c07-4d04-8560-a613d4c44c0a None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:18.255596 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63b5c2a-699a-4fe2-9268-e9476f068c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:18.256390 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63b5c2a-699a-4fe2-9268-e9476f068c76 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:18.256943 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63b5c2a-699a-4fe2-9268-e9476f068c76 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:18.257315 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63b5c2a-699a-4fe2-9268-e9476f068c76 None admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:18.258201 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63b5c2a-699a-4fe2-9268-e9476f068c76 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:18.258607 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63b5c2a-699a-4fe2-9268-e9476f068c76 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:18.260124 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63b5c2a-699a-4fe2-9268-e9476f068c76 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:18.278093 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 439/857] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Jun 16 17:31:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:31:18.279146 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63b5c2a-699a-4fe2-9268-e9476f068c76 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:18.294399 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b15d090-79c7-4608-a7b3-4d6785d1bc5a None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:18.345271 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b91e5fc-9eb6-4f47-a301-0e12ea99e3a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:18.345951 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b91e5fc-9eb6-4f47-a301-0e12ea99e3a2 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:18.346231 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b91e5fc-9eb6-4f47-a301-0e12ea99e3a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:18.346504 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b91e5fc-9eb6-4f47-a301-0e12ea99e3a2 None admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:18.347338 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b91e5fc-9eb6-4f47-a301-0e12ea99e3a2 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:18.347649 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b91e5fc-9eb6-4f47-a301-0e12ea99e3a2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:18.349792 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b91e5fc-9eb6-4f47-a301-0e12ea99e3a2 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:18.373990 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 419/858] 10.222.0.42 () {56 vars in 1135 bytes} [Thu Jun 16 17:31:18 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:31:18.379306 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b91e5fc-9eb6-4f47-a301-0e12ea99e3a2 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:18.424574 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63b5c2a-699a-4fe2-9268-e9476f068c76 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:18.467830 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9617e2-d808-4a46-92ef-ef0ae981ff44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:18.469641 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9617e2-d808-4a46-92ef-ef0ae981ff44 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:18.470385 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9617e2-d808-4a46-92ef-ef0ae981ff44 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:18.470939 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9617e2-d808-4a46-92ef-ef0ae981ff44 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:18.472010 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9617e2-d808-4a46-92ef-ef0ae981ff44 None admin] RBAC: Adding query filter params (role.id=cd7dd29fcdc844c78544402dc30986fb, scope.project.id=8abb87f2f141451a8c6ed832db82be18, user.id=71443c30b4554719b5549493bf3cc00c) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:18.472518 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9617e2-d808-4a46-92ef-ef0ae981ff44 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:18.474262 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9617e2-d808-4a46-92ef-ef0ae981ff44 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:18.479347 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:18.489224 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 440/859] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jun 16 17:31:18 2022] GET /identity/v3/role_assignments?user.id=71443c30b4554719b5549493bf3cc00c&scope.project.id=8abb87f2f141451a8c6ed832db82be18&role.id=cd7dd29fcdc844c78544402dc30986fb => generated 608 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:18.518044 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b91e5fc-9eb6-4f47-a301-0e12ea99e3a2 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:18.555725 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6648e17e-dc50-4e8d-ab4e-0d1bde1f9bd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:18.556414 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6648e17e-dc50-4e8d-ab4e-0d1bde1f9bd6 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:18.556710 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6648e17e-dc50-4e8d-ab4e-0d1bde1f9bd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:18.556988 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6648e17e-dc50-4e8d-ab4e-0d1bde1f9bd6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:18.643731 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6648e17e-dc50-4e8d-ab4e-0d1bde1f9bd6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:18.645679 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6648e17e-dc50-4e8d-ab4e-0d1bde1f9bd6 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:18.728666 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:18.766855 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 420/860] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jun 16 17:31:18 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 17:31:18.772468 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9617e2-d808-4a46-92ef-ef0ae981ff44 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:18.785965 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:18.896967 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9617e2-d808-4a46-92ef-ef0ae981ff44 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:18.937724 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32831182-f4a1-40f8-87d2-7ef99d74b24f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:18.938788 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32831182-f4a1-40f8-87d2-7ef99d74b24f None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:18.939212 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32831182-f4a1-40f8-87d2-7ef99d74b24f None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:18.939624 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32831182-f4a1-40f8-87d2-7ef99d74b24f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:18.951330 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32831182-f4a1-40f8-87d2-7ef99d74b24f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:18.953715 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32831182-f4a1-40f8-87d2-7ef99d74b24f None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:18.972317 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-32831182-f4a1-40f8-87d2-7ef99d74b24f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:31:18.972317 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:18.972317 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:31:18.972317 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:31:18.972317 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:31:18.972317 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:31:18.972317 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:31:18.972317 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:31:18.972317 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:31:18.972317 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:31:18.972317 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:31:18.972317 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:31:18.972317 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:31:18.972317 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:31:18.972317 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:31:18.973438 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:31:18.973438 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:31:18.973438 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:31:18.973438 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:31:18.973438 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:31:18.973438 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:31:18.973438 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:31:18.973438 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:31:18.973438 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:31:18.973438 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:31:18.973438 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:31:18.973438 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:31:18.973438 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:31:18.973438 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:31:18.973438 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:31:18.973438 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:31:18.973438 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:31:18.974498 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:31:18.974498 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:31:18.974498 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:31:18.974498 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:31:18.974843 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1435aae3-2c19-4e95-876b-3ef53b0409b6 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:18.975265 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1435aae3-2c19-4e95-876b-3ef53b0409b6 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:18.975396 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1435aae3-2c19-4e95-876b-3ef53b0409b6 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:18.977003 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 441/862] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jun 16 17:31:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:31:18.980479 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 421/862] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:31:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:31:18.982538 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56000e4e-95b0-4a78-8fed-eabb61fbdae2 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:18.982833 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56000e4e-95b0-4a78-8fed-eabb61fbdae2 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:18.982998 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1435aae3-2c19-4e95-876b-3ef53b0409b6 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:18.983102 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56000e4e-95b0-4a78-8fed-eabb61fbdae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:19.001674 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-56000e4e-95b0-4a78-8fed-eabb61fbdae2 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:31:19.002878 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-56000e4e-95b0-4a78-8fed-eabb61fbdae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:31:19.063270 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:19.084512 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 442/863] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:31:18 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:31:19.102334 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56000e4e-95b0-4a78-8fed-eabb61fbdae2 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:19.112058 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1435aae3-2c19-4e95-876b-3ef53b0409b6 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:19.157257 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ea6df5-fa3d-461e-873b-cf5506823b3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:19.158276 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ea6df5-fa3d-461e-873b-cf5506823b3b None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:19.158407 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ea6df5-fa3d-461e-873b-cf5506823b3b None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:19.158605 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ea6df5-fa3d-461e-873b-cf5506823b3b None admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:19.159341 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ea6df5-fa3d-461e-873b-cf5506823b3b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:19.159648 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ea6df5-fa3d-461e-873b-cf5506823b3b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:19.161267 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ea6df5-fa3d-461e-873b-cf5506823b3b None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:19.180669 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 422/864] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jun 16 17:31:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:31:19.182129 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ea6df5-fa3d-461e-873b-cf5506823b3b None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:19.223510 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:19.230254 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56000e4e-95b0-4a78-8fed-eabb61fbdae2 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:19.269639 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75179edd-b270-4acb-aa7a-dcdfff99cce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:19.270514 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75179edd-b270-4acb-aa7a-dcdfff99cce1 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:19.270799 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75179edd-b270-4acb-aa7a-dcdfff99cce1 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:19.271072 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75179edd-b270-4acb-aa7a-dcdfff99cce1 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:19.271733 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75179edd-b270-4acb-aa7a-dcdfff99cce1 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:19.273494 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75179edd-b270-4acb-aa7a-dcdfff99cce1 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:19.288176 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-75179edd-b270-4acb-aa7a-dcdfff99cce1 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 17:31:19.288176 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:19.288176 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 17:31:19.288176 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 17:31:19.288176 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 17:31:19.288176 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 17:31:19.288176 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 17:31:19.288176 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:31:19.288176 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 17:31:19.288176 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:31:19.288176 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:31:19.288176 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:31:19.288176 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:19.288176 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:31:19.288176 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:31:19.288176 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:31:19.289160 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:31:19.289160 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:31:19.289160 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:31:19.289160 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:31:19.289160 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:31:19.289160 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:31:19.289160 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:31:19.289160 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 17:31:19.289160 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 17:31:19.289160 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 17:31:19.289160 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 17:31:19.289160 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:31:19.289160 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:31:19.289160 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:31:19.289160 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:31:19.289160 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:31:19.290001 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:31:19.290001 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:31:19.290001 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:31:19.290001 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:31:19.290001 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:31:19.290001 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:31:19.290001 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:31:19.290001 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:31:19.290001 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:31:19.290001 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:31:19.290001 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:31:19.290001 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 17:31:19.290001 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:31:19.290001 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 17:31:19.290001 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:31:19.294345 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 443/865] 10.222.0.42 () {56 vars in 1132 bytes} [Thu Jun 16 17:31:19 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:31:19.295679 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75179edd-b270-4acb-aa7a-dcdfff99cce1 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:19.305290 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ea6df5-fa3d-461e-873b-cf5506823b3b None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:19.342422 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2a19ec-e740-4592-b4f9-5facaa32c678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:19.343626 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2a19ec-e740-4592-b4f9-5facaa32c678 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:19.344118 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2a19ec-e740-4592-b4f9-5facaa32c678 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:19.344604 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2a19ec-e740-4592-b4f9-5facaa32c678 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:19.352756 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2a19ec-e740-4592-b4f9-5facaa32c678 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:19.354566 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2a19ec-e740-4592-b4f9-5facaa32c678 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:19.363347 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:19.371063 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-6f2a19ec-e740-4592-b4f9-5facaa32c678 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:31:19.371063 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:19.371063 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:31:19.371063 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:31:19.371063 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:31:19.371063 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:31:19.371063 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:31:19.371063 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:31:19.371063 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:31:19.371063 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:31:19.371063 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:31:19.371063 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:31:19.371063 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:31:19.371063 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:31:19.371063 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:31:19.372029 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:31:19.372029 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:31:19.372029 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:31:19.372029 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:31:19.372029 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:31:19.372029 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:31:19.372029 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:31:19.372029 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:31:19.372029 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:31:19.372029 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:31:19.372029 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:31:19.372029 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:31:19.372029 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:31:19.372029 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:31:19.372029 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:31:19.372029 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:31:19.372029 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:31:19.372984 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:31:19.372984 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:31:19.372984 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:31:19.372984 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:31:19.372984 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:31:19.372984 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:31:19.372984 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:31:19.372984 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:31:19.380544 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 423/866] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jun 16 17:31:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:31:19.381770 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2a19ec-e740-4592-b4f9-5facaa32c678 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:19.417954 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75179edd-b270-4acb-aa7a-dcdfff99cce1 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:19.454416 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8c7794-567f-4b76-b96d-8aee8912294b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:19.455093 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8c7794-567f-4b76-b96d-8aee8912294b None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:19.455379 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8c7794-567f-4b76-b96d-8aee8912294b None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:19.455659 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8c7794-567f-4b76-b96d-8aee8912294b None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:19.456364 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8c7794-567f-4b76-b96d-8aee8912294b None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:19.456698 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8c7794-567f-4b76-b96d-8aee8912294b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:19.458069 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8c7794-567f-4b76-b96d-8aee8912294b None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:19.462724 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 444/867] 10.222.0.42 () {56 vars in 1126 bytes} [Thu Jun 16 17:31:19 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:31:19.464065 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95204872-2912-4c1d-9e3a-7431c6eb5017 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:19.464357 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95204872-2912-4c1d-9e3a-7431c6eb5017 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:19.464664 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95204872-2912-4c1d-9e3a-7431c6eb5017 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:19.466086 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 445/868] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:31:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:31:19.471356 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95204872-2912-4c1d-9e3a-7431c6eb5017 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:19.487613 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:19.500747 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2a19ec-e740-4592-b4f9-5facaa32c678 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:19.537922 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680cded3-486e-4d33-a576-4f230d1af783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:19.539043 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680cded3-486e-4d33-a576-4f230d1af783 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:19.539043 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680cded3-486e-4d33-a576-4f230d1af783 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:19.539534 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680cded3-486e-4d33-a576-4f230d1af783 None admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:19.540017 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680cded3-486e-4d33-a576-4f230d1af783 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:19.540328 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680cded3-486e-4d33-a576-4f230d1af783 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:19.541865 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680cded3-486e-4d33-a576-4f230d1af783 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:19.554726 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 424/869] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Jun 16 17:31:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:19.556064 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aa2d5a-6c3a-4085-b989-f8b80c8b8750 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:19.556355 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aa2d5a-6c3a-4085-b989-f8b80c8b8750 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:19.556662 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aa2d5a-6c3a-4085-b989-f8b80c8b8750 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:19.574323 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-13aa2d5a-6c3a-4085-b989-f8b80c8b8750 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:31:19.575528 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-13aa2d5a-6c3a-4085-b989-f8b80c8b8750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:31:19.588944 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95204872-2912-4c1d-9e3a-7431c6eb5017 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:19.610669 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:19.630809 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6714645b-6db2-4013-a8d2-549a65387b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:19.631346 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6714645b-6db2-4013-a8d2-549a65387b58 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:19.631628 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6714645b-6db2-4013-a8d2-549a65387b58 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:19.631900 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6714645b-6db2-4013-a8d2-549a65387b58 None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:19.632638 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6714645b-6db2-4013-a8d2-549a65387b58 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:19.632948 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6714645b-6db2-4013-a8d2-549a65387b58 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:19.634310 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6714645b-6db2-4013-a8d2-549a65387b58 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:19.638861 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 446/870] 10.222.0.42 () {56 vars in 1126 bytes} [Thu Jun 16 17:31:19 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:19.639637 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6714645b-6db2-4013-a8d2-549a65387b58 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:19.663502 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 425/871] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:31:19 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:31:19.689162 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13aa2d5a-6c3a-4085-b989-f8b80c8b8750 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:19.797193 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6714645b-6db2-4013-a8d2-549a65387b58 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:19.805759 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:19.830726 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13aa2d5a-6c3a-4085-b989-f8b80c8b8750 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:19.846515 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f60b398-f834-436e-8da4-c3b8bb51c13b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:19.847185 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f60b398-f834-436e-8da4-c3b8bb51c13b None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:19.847473 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f60b398-f834-436e-8da4-c3b8bb51c13b None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:19.847749 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f60b398-f834-436e-8da4-c3b8bb51c13b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:19.848635 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f60b398-f834-436e-8da4-c3b8bb51c13b None admin] RBAC: Adding query filter params (role.id=3ca2dcbbd29d4a31a3ab78f2299bc239, scope.project.id=8abb87f2f141451a8c6ed832db82be18, user.id=47b7a9cd615c41d6983a0007b475038c) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:19.848950 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f60b398-f834-436e-8da4-c3b8bb51c13b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:19.850389 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f60b398-f834-436e-8da4-c3b8bb51c13b None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:19.858535 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 447/872] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jun 16 17:31:19 2022] GET /identity/v3/role_assignments?user.id=47b7a9cd615c41d6983a0007b475038c&scope.project.id=8abb87f2f141451a8c6ed832db82be18&role.id=3ca2dcbbd29d4a31a3ab78f2299bc239 => generated 608 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:19.879131 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146b6a2c-8638-4720-8ca1-cfde7fd40bb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:19.879717 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146b6a2c-8638-4720-8ca1-cfde7fd40bb6 None admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:19.880020 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146b6a2c-8638-4720-8ca1-cfde7fd40bb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:19.880303 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146b6a2c-8638-4720-8ca1-cfde7fd40bb6 None admin] PATH_INFO: `/v3/services` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:19.881099 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146b6a2c-8638-4720-8ca1-cfde7fd40bb6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:19.882537 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146b6a2c-8638-4720-8ca1-cfde7fd40bb6 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:19.904315 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 426/873] 10.222.0.42 () {60 vars in 1141 bytes} [Thu Jun 16 17:31:19 2022] POST /identity/v3/services => generated 251 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:31:20.870884 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8783d529-3ba8-40a7-b6f2-26757179e61d None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:20.871757 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8783d529-3ba8-40a7-b6f2-26757179e61d None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:20.872197 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8783d529-3ba8-40a7-b6f2-26757179e61d None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:20.873946 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 448/874] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:31:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:31:20.880191 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e33145-0ce4-4579-878d-400d2cf3f93c None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:20.881092 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e33145-0ce4-4579-878d-400d2cf3f93c None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:20.881583 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e33145-0ce4-4579-878d-400d2cf3f93c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:20.900398 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-d1e33145-0ce4-4579-878d-400d2cf3f93c None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:31:20.901888 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-d1e33145-0ce4-4579-878d-400d2cf3f93c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:31:20.984859 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 427/875] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:31:20 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:31:21.010764 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8783d529-3ba8-40a7-b6f2-26757179e61d None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:21.067963 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:21.129036 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8783d529-3ba8-40a7-b6f2-26757179e61d None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:21.166471 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e1f941-97c6-49a7-8e8e-52621aa80978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:21.167346 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e1f941-97c6-49a7-8e8e-52621aa80978 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:21.167633 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e1f941-97c6-49a7-8e8e-52621aa80978 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:21.167912 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e1f941-97c6-49a7-8e8e-52621aa80978 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:21.179854 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e1f941-97c6-49a7-8e8e-52621aa80978 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:31:21.188900 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-c4e1f941-97c6-49a7-8e8e-52621aa80978 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 17:31:21.188900 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:21.188900 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:31:21.188900 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:31:21.188900 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:31:21.188900 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:31:21.188900 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:31:21.188900 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:31:21.188900 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:31:21.188900 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:31:21.188900 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:31:21.188900 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:31:21.188900 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:31:21.188900 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:31:21.188900 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:31:21.188900 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:31:21.189942 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:31:21.189942 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:31:21.189942 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:31:21.189942 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:31:21.189942 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:31:21.189942 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:31:21.189942 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:31:21.189942 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:31:21.189942 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:31:21.189942 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:31:21.189942 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:31:21.189942 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:31:21.189942 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:31:21.189942 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:31:21.189942 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:31:21.189942 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:31:21.190788 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:31:21.190788 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:31:21.190788 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:31:21.190788 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:31:21.190788 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:31:21.190788 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:31:21.190788 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:31:21.190788 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:31:21.190788 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:31:21.190788 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:31:21.190788 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 17:31:21.190788 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:31:21.190788 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 449/876] 10.222.0.42 () {56 vars in 1120 bytes} [Thu Jun 16 17:31:21 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:31:21.197910 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e33145-0ce4-4579-878d-400d2cf3f93c None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:21.272658 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:21.308919 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e33145-0ce4-4579-878d-400d2cf3f93c None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:21.347767 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a6d399-dc70-4ab8-8e66-da91f8dbff17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:21.349066 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a6d399-dc70-4ab8-8e66-da91f8dbff17 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:21.349549 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a6d399-dc70-4ab8-8e66-da91f8dbff17 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:21.350041 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a6d399-dc70-4ab8-8e66-da91f8dbff17 None admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:21.350998 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a6d399-dc70-4ab8-8e66-da91f8dbff17 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:21.351512 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a6d399-dc70-4ab8-8e66-da91f8dbff17 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:21.353325 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a6d399-dc70-4ab8-8e66-da91f8dbff17 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:21.364227 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 428/877] 10.222.0.42 () {56 vars in 1115 bytes} [Thu Jun 16 17:31:21 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:21.369999 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e1f941-97c6-49a7-8e8e-52621aa80978 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:21.480976 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e1f941-97c6-49a7-8e8e-52621aa80978 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:21.492829 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:21.518736 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f210b135-1c1b-45d2-ad8d-2b582f593989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:21.519535 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f210b135-1c1b-45d2-ad8d-2b582f593989 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:21.519822 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f210b135-1c1b-45d2-ad8d-2b582f593989 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:21.520100 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f210b135-1c1b-45d2-ad8d-2b582f593989 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:21.532135 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f210b135-1c1b-45d2-ad8d-2b582f593989 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:21.533684 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f210b135-1c1b-45d2-ad8d-2b582f593989 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:21.551389 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-f210b135-1c1b-45d2-ad8d-2b582f593989 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:31:21.551389 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:21.551389 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:31:21.551389 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:31:21.551389 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:31:21.551389 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:31:21.551389 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:31:21.551389 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:31:21.551389 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:31:21.551389 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:31:21.551389 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:31:21.551389 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:31:21.551389 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:31:21.551389 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:31:21.551389 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:31:21.552331 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:31:21.552331 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:31:21.552331 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:31:21.552331 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:31:21.552331 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:31:21.552331 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:31:21.552331 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:31:21.552331 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:31:21.552331 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:31:21.552331 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:31:21.552331 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:31:21.552331 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:31:21.552331 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:31:21.552331 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:31:21.552331 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:31:21.552331 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:31:21.552331 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:31:21.553253 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:31:21.553253 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:31:21.553253 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:31:21.553253 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:31:21.553253 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 450/878] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jun 16 17:31:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:31:21.559565 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a6d399-dc70-4ab8-8e66-da91f8dbff17 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:21.667601 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a6d399-dc70-4ab8-8e66-da91f8dbff17 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:21.700799 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:21.706686 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355698ff-8087-44a8-a0f0-c5ef3983c548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:21.707850 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355698ff-8087-44a8-a0f0-c5ef3983c548 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:21.708353 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355698ff-8087-44a8-a0f0-c5ef3983c548 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:21.708884 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355698ff-8087-44a8-a0f0-c5ef3983c548 None admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:21.709827 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355698ff-8087-44a8-a0f0-c5ef3983c548 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:21.710353 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355698ff-8087-44a8-a0f0-c5ef3983c548 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:21.711980 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355698ff-8087-44a8-a0f0-c5ef3983c548 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:21.726791 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 429/879] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jun 16 17:31:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:21.734387 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f210b135-1c1b-45d2-ad8d-2b582f593989 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:21.857987 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f210b135-1c1b-45d2-ad8d-2b582f593989 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:21.910050 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12d3215-3015-4c1d-8c2d-5c5c5efb1f93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:21.910880 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12d3215-3015-4c1d-8c2d-5c5c5efb1f93 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:21.911164 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12d3215-3015-4c1d-8c2d-5c5c5efb1f93 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:21.911441 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12d3215-3015-4c1d-8c2d-5c5c5efb1f93 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:21.919799 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12d3215-3015-4c1d-8c2d-5c5c5efb1f93 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:21.921970 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12d3215-3015-4c1d-8c2d-5c5c5efb1f93 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:21.929002 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39110d24-fd14-436b-bfac-248896f335ba None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:21.929469 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39110d24-fd14-436b-bfac-248896f335ba None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:21.929566 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39110d24-fd14-436b-bfac-248896f335ba None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:21.931140 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 430/880] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:31:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:31:21.936032 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8de2848-c5b4-4ceb-9de4-dae1c3ca4638 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:21.936317 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8de2848-c5b4-4ceb-9de4-dae1c3ca4638 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:21.936637 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8de2848-c5b4-4ceb-9de4-dae1c3ca4638 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:21.938381 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-c12d3215-3015-4c1d-8c2d-5c5c5efb1f93 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 17:31:21.938381 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:21.938381 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:31:21.938381 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:31:21.938381 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:31:21.938381 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:31:21.938381 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:31:21.938381 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:31:21.938381 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:31:21.938381 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:31:21.938381 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:31:21.938381 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:31:21.938381 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:31:21.938381 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:31:21.938381 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:31:21.938381 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:31:21.939292 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:31:21.939292 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:31:21.939292 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:31:21.939292 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:31:21.939292 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:31:21.939292 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:31:21.939292 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:31:21.939292 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:31:21.939292 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:31:21.939292 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:31:21.939292 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:31:21.939292 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:31:21.939292 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:31:21.939292 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:31:21.939292 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:31:21.939292 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:31:21.939292 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:31:21.939292 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:31:21.940241 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:31:21.940241 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:31:21.940241 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:31:21.940241 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:31:21.940241 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:31:21.940241 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:31:21.940241 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:31:21.940241 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:31:21.940241 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 17:31:21.940241 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:31:21.940241 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 451/881] 10.222.0.42 () {56 vars in 1117 bytes} [Thu Jun 16 17:31:21 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:31:21.945867 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12d3215-3015-4c1d-8c2d-5c5c5efb1f93 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:21.964205 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-f8de2848-c5b4-4ceb-9de4-dae1c3ca4638 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:31:21.965841 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-f8de2848-c5b4-4ceb-9de4-dae1c3ca4638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:31:21.966348 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:22.043777 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 431/882] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:31:21 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 17:31:22.052532 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12d3215-3015-4c1d-8c2d-5c5c5efb1f93 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:22.066709 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8de2848-c5b4-4ceb-9de4-dae1c3ca4638 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:22.089499 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11d30b8-ba18-4c6b-b9cc-cdd9f0702402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:22.090342 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11d30b8-ba18-4c6b-b9cc-cdd9f0702402 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:22.090630 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11d30b8-ba18-4c6b-b9cc-cdd9f0702402 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:22.090907 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11d30b8-ba18-4c6b-b9cc-cdd9f0702402 None admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:22.091764 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11d30b8-ba18-4c6b-b9cc-cdd9f0702402 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:22.092075 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11d30b8-ba18-4c6b-b9cc-cdd9f0702402 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:22.094452 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11d30b8-ba18-4c6b-b9cc-cdd9f0702402 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:22.120566 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 452/883] 10.222.0.42 () {56 vars in 1149 bytes} [Thu Jun 16 17:31:21 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:31:22.122509 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11d30b8-ba18-4c6b-b9cc-cdd9f0702402 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:22.219654 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:22.246181 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11d30b8-ba18-4c6b-b9cc-cdd9f0702402 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:22.283487 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd896fea-a428-47f3-99e5-6bb7826d75e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:22.284146 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd896fea-a428-47f3-99e5-6bb7826d75e7 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:22.284458 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd896fea-a428-47f3-99e5-6bb7826d75e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:22.284740 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd896fea-a428-47f3-99e5-6bb7826d75e7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:22.309982 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8de2848-c5b4-4ceb-9de4-dae1c3ca4638 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:22.331337 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:22.346428 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e613f92-cbd6-4d09-aa14-a8c13f0f68b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:22.347091 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e613f92-cbd6-4d09-aa14-a8c13f0f68b2 None admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:22.347372 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e613f92-cbd6-4d09-aa14-a8c13f0f68b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:22.347648 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e613f92-cbd6-4d09-aa14-a8c13f0f68b2 None admin] PATH_INFO: `/v3/services` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:22.348351 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e613f92-cbd6-4d09-aa14-a8c13f0f68b2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:22.349740 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e613f92-cbd6-4d09-aa14-a8c13f0f68b2 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:22.353610 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd896fea-a428-47f3-99e5-6bb7826d75e7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:22.355039 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd896fea-a428-47f3-99e5-6bb7826d75e7 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:22.363004 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 432/884] 10.222.0.42 () {60 vars in 1141 bytes} [Thu Jun 16 17:31:22 2022] POST /identity/v3/services => generated 269 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:31:22.364308 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91582555-57f4-4cce-98fa-22c7b0569172 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:22.364633 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91582555-57f4-4cce-98fa-22c7b0569172 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:22.364912 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91582555-57f4-4cce-98fa-22c7b0569172 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:22.366166 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 433/885] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:31:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:31:22.367278 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d468a044-b6f3-41cd-a880-3b592c41aabd None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:22.367544 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d468a044-b6f3-41cd-a880-3b592c41aabd None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:22.367821 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d468a044-b6f3-41cd-a880-3b592c41aabd None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:22.373660 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 434/886] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:31:22 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:31:22.375634 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1385ea0e-4fa3-4661-9e87-a170d251f95b None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:22.375979 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1385ea0e-4fa3-4661-9e87-a170d251f95b None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:22.376201 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1385ea0e-4fa3-4661-9e87-a170d251f95b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:22.394882 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-1385ea0e-4fa3-4661-9e87-a170d251f95b None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:31:22.396122 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-1385ea0e-4fa3-4661-9e87-a170d251f95b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:31:22.470928 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:22.495393 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 435/887] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:31:22 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:31:22.506357 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a013c0e9-21ef-4e67-afbf-e5e51883da6b None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:22.506357 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a013c0e9-21ef-4e67-afbf-e5e51883da6b None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:22.506357 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a013c0e9-21ef-4e67-afbf-e5e51883da6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:22.506716 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 453/888] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jun 16 17:31:22 2022] GET /identity/v3/auth/tokens => generated 1252 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 16 17:31:22.508673 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd896fea-a428-47f3-99e5-6bb7826d75e7 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:22.525176 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-a013c0e9-21ef-4e67-afbf-e5e51883da6b None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:31:22.526406 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-a013c0e9-21ef-4e67-afbf-e5e51883da6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:31:22.607080 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 436/889] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:31:22 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:31:22.607943 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a013c0e9-21ef-4e67-afbf-e5e51883da6b None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:22.632520 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd896fea-a428-47f3-99e5-6bb7826d75e7 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:22.652455 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:22.670776 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417aa147-30d8-48fc-8a88-078f89e275a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:22.672927 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417aa147-30d8-48fc-8a88-078f89e275a2 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:22.673218 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417aa147-30d8-48fc-8a88-078f89e275a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:22.673494 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417aa147-30d8-48fc-8a88-078f89e275a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:22.684846 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417aa147-30d8-48fc-8a88-078f89e275a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:22.686300 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417aa147-30d8-48fc-8a88-078f89e275a2 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:22.707838 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-417aa147-30d8-48fc-8a88-078f89e275a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:31:22.707838 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:22.707838 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:31:22.707838 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:31:22.707838 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:31:22.707838 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:31:22.707838 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:31:22.707838 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:31:22.707838 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:31:22.707838 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:31:22.707838 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:31:22.707838 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:31:22.707838 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:31:22.707838 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:31:22.707838 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:31:22.708998 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:31:22.708998 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:31:22.708998 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:31:22.708998 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:31:22.708998 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:31:22.708998 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:31:22.708998 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:31:22.708998 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:31:22.708998 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:31:22.708998 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:31:22.708998 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:31:22.708998 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:31:22.708998 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:31:22.708998 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:31:22.708998 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:31:22.708998 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:31:22.708998 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:31:22.709922 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:31:22.709922 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:31:22.709922 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:31:22.709922 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:31:22.709922 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 454/890] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jun 16 17:31:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:31:22.710216 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417aa147-30d8-48fc-8a88-078f89e275a2 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:22.727371 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a013c0e9-21ef-4e67-afbf-e5e51883da6b None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:22.744536 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:22.766749 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4b275a-cf58-4381-b24d-6305d228fdb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:22.767563 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4b275a-cf58-4381-b24d-6305d228fdb5 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:22.767839 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4b275a-cf58-4381-b24d-6305d228fdb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:22.768393 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4b275a-cf58-4381-b24d-6305d228fdb5 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:22.768804 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4b275a-cf58-4381-b24d-6305d228fdb5 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:22.770172 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4b275a-cf58-4381-b24d-6305d228fdb5 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:22.782635 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-cf4b275a-cf58-4381-b24d-6305d228fdb5 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 17:31:22.782635 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:22.782635 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 17:31:22.782635 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 17:31:22.782635 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 17:31:22.782635 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 17:31:22.782635 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 17:31:22.782635 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:31:22.782635 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 17:31:22.782635 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:31:22.782635 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:31:22.782635 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:31:22.782635 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:22.782635 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:31:22.782635 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:31:22.782635 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:31:22.782635 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:31:22.783698 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:31:22.783698 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:31:22.783698 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:31:22.783698 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:31:22.783698 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:31:22.783698 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:31:22.783698 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 17:31:22.783698 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 17:31:22.783698 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 17:31:22.783698 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 17:31:22.783698 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:31:22.783698 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:31:22.783698 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:31:22.783698 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:31:22.783698 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:31:22.783698 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:31:22.784579 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:31:22.784579 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:31:22.784579 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:31:22.784579 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:31:22.784579 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:31:22.784579 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:31:22.784579 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:31:22.784579 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:31:22.784579 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:31:22.784579 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:31:22.784579 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 17:31:22.784579 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:31:22.784579 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 17:31:22.784579 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:31:22.784579 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 437/891] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Jun 16 17:31:22 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:31:22.785373 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4b275a-cf58-4381-b24d-6305d228fdb5 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:22.831080 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417aa147-30d8-48fc-8a88-078f89e275a2 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:22.868059 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce9b492-6ae8-4100-abb5-46ab108e665e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:22.868859 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce9b492-6ae8-4100-abb5-46ab108e665e None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:22.869156 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce9b492-6ae8-4100-abb5-46ab108e665e None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:22.869438 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce9b492-6ae8-4100-abb5-46ab108e665e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:22.870098 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce9b492-6ae8-4100-abb5-46ab108e665e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:22.871484 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce9b492-6ae8-4100-abb5-46ab108e665e None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:22.885510 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-fce9b492-6ae8-4100-abb5-46ab108e665e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 17:31:22.885510 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:22.885510 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 17:31:22.885510 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 17:31:22.885510 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 17:31:22.885510 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 17:31:22.885510 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 17:31:22.885510 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:31:22.885510 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 17:31:22.885510 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:31:22.885510 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:31:22.885510 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:31:22.885510 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:22.885510 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:31:22.885510 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:31:22.885510 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:31:22.885510 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:31:22.886801 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:31:22.886801 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:31:22.886801 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:31:22.886801 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:31:22.886801 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:31:22.886801 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:31:22.886801 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 17:31:22.886801 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 17:31:22.886801 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 17:31:22.886801 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 17:31:22.886801 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:31:22.886801 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:31:22.886801 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:31:22.886801 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:31:22.886801 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:31:22.886801 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:31:22.888336 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:31:22.888336 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:31:22.888336 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:31:22.888336 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:31:22.888336 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:31:22.888336 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:31:22.888336 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:31:22.888336 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:31:22.888336 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:31:22.888336 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:31:22.888336 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 17:31:22.888336 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:31:22.888336 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 17:31:22.888336 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:31:22.891380 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 455/892] 10.222.0.42 () {56 vars in 1126 bytes} [Thu Jun 16 17:31:22 2022] GET /identity/v3/services/object-store => generated 93 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:31:22.892129 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce9b492-6ae8-4100-abb5-46ab108e665e None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:22.901961 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:22.905803 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4b275a-cf58-4381-b24d-6305d228fdb5 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:22.943003 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec58cb0f-c43d-449c-be36-aed1553185a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:22.943696 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec58cb0f-c43d-449c-be36-aed1553185a6 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:22.943942 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec58cb0f-c43d-449c-be36-aed1553185a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:22.944218 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec58cb0f-c43d-449c-be36-aed1553185a6 None admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:22.944971 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec58cb0f-c43d-449c-be36-aed1553185a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:22.945284 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec58cb0f-c43d-449c-be36-aed1553185a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:22.946662 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec58cb0f-c43d-449c-be36-aed1553185a6 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:22.965350 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 438/893] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jun 16 17:31:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:22.965350 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec58cb0f-c43d-449c-be36-aed1553185a6 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:23.000039 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:23.012884 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce9b492-6ae8-4100-abb5-46ab108e665e None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:23.050812 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7c363d-662e-466c-9087-bf40e396f226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:23.051479 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7c363d-662e-466c-9087-bf40e396f226 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:23.051761 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7c363d-662e-466c-9087-bf40e396f226 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:23.052033 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7c363d-662e-466c-9087-bf40e396f226 None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:23.052768 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7c363d-662e-466c-9087-bf40e396f226 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:23.053269 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7c363d-662e-466c-9087-bf40e396f226 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:23.054427 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7c363d-662e-466c-9087-bf40e396f226 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:23.059584 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 456/894] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Jun 16 17:31:22 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:23.061576 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7c363d-662e-466c-9087-bf40e396f226 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:23.078452 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec58cb0f-c43d-449c-be36-aed1553185a6 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:23.116630 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da37fdb2-9591-48b3-8198-9ddad191ea2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:23.117294 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da37fdb2-9591-48b3-8198-9ddad191ea2f None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:23.117572 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da37fdb2-9591-48b3-8198-9ddad191ea2f None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:23.117842 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da37fdb2-9591-48b3-8198-9ddad191ea2f None admin] PATH_INFO: `/v3/services` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:23.118587 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da37fdb2-9591-48b3-8198-9ddad191ea2f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:23.118942 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da37fdb2-9591-48b3-8198-9ddad191ea2f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:23.120291 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da37fdb2-9591-48b3-8198-9ddad191ea2f None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:23.130328 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 439/895] 10.222.0.42 () {56 vars in 1122 bytes} [Thu Jun 16 17:31:22 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:23.130328 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da37fdb2-9591-48b3-8198-9ddad191ea2f None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:23.163530 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:23.188049 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7c363d-662e-466c-9087-bf40e396f226 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:23.229142 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcb9eaa-6e94-47f9-bda9-c236e3b67fd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:23.229928 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcb9eaa-6e94-47f9-bda9-c236e3b67fd6 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:23.230220 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcb9eaa-6e94-47f9-bda9-c236e3b67fd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:23.230488 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcb9eaa-6e94-47f9-bda9-c236e3b67fd6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:23.238384 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcb9eaa-6e94-47f9-bda9-c236e3b67fd6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:23.239824 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcb9eaa-6e94-47f9-bda9-c236e3b67fd6 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:23.256259 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-2dcb9eaa-6e94-47f9-bda9-c236e3b67fd6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:31:23.256259 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:23.256259 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:31:23.256259 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:31:23.256259 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:31:23.256259 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:31:23.256259 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:31:23.256259 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:31:23.256259 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:31:23.256259 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:31:23.256259 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:31:23.256259 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:31:23.256259 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:31:23.256259 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:31:23.256259 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:31:23.257212 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:31:23.257212 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:31:23.257212 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:31:23.257212 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:31:23.257212 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:31:23.257212 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:31:23.257212 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:31:23.257212 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:31:23.257212 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:31:23.257212 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:31:23.257212 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:31:23.257212 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:31:23.257212 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:31:23.257212 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:31:23.257212 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:31:23.257212 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:31:23.257212 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:31:23.258112 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:31:23.258112 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:31:23.258112 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:31:23.258112 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:31:23.258112 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:31:23.258112 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:31:23.258112 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:31:23.258112 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:31:23.258551 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da37fdb2-9591-48b3-8198-9ddad191ea2f None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:23.258626 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 457/896] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jun 16 17:31:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:31:23.258626 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcb9eaa-6e94-47f9-bda9-c236e3b67fd6 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:23.277751 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:23.303312 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becf3baa-3978-45b8-b25a-5e2e9e6f0c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:23.303979 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becf3baa-3978-45b8-b25a-5e2e9e6f0c13 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:23.304257 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becf3baa-3978-45b8-b25a-5e2e9e6f0c13 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:23.304557 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becf3baa-3978-45b8-b25a-5e2e9e6f0c13 None admin] PATH_INFO: `/v3/services` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:23.305623 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becf3baa-3978-45b8-b25a-5e2e9e6f0c13 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:23.305925 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becf3baa-3978-45b8-b25a-5e2e9e6f0c13 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:23.307590 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becf3baa-3978-45b8-b25a-5e2e9e6f0c13 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:23.313296 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 440/897] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Jun 16 17:31:23 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:23.314103 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becf3baa-3978-45b8-b25a-5e2e9e6f0c13 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:23.372897 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcb9eaa-6e94-47f9-bda9-c236e3b67fd6 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:23.409085 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:23.420445 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fb45a4-1d4c-4839-b772-dbc3c60556ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:23.421939 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fb45a4-1d4c-4839-b772-dbc3c60556ab None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:23.422062 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becf3baa-3978-45b8-b25a-5e2e9e6f0c13 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:23.422479 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fb45a4-1d4c-4839-b772-dbc3c60556ab None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:23.422921 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fb45a4-1d4c-4839-b772-dbc3c60556ab None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:23.423924 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fb45a4-1d4c-4839-b772-dbc3c60556ab None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:23.424638 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fb45a4-1d4c-4839-b772-dbc3c60556ab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:23.425951 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fb45a4-1d4c-4839-b772-dbc3c60556ab None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:23.440415 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 458/898] 10.222.0.42 () {56 vars in 1122 bytes} [Thu Jun 16 17:31:23 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:23.440415 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fb45a4-1d4c-4839-b772-dbc3c60556ab None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:23.470834 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8e10c5-9b17-4c8d-a117-b8b38bf540a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:23.471578 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8e10c5-9b17-4c8d-a117-b8b38bf540a9 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:23.471863 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8e10c5-9b17-4c8d-a117-b8b38bf540a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:23.472133 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8e10c5-9b17-4c8d-a117-b8b38bf540a9 None admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:23.473157 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8e10c5-9b17-4c8d-a117-b8b38bf540a9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:23.473459 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8e10c5-9b17-4c8d-a117-b8b38bf540a9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:23.475158 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8e10c5-9b17-4c8d-a117-b8b38bf540a9 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:23.490881 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 441/899] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Jun 16 17:31:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:23.496782 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8e10c5-9b17-4c8d-a117-b8b38bf540a9 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:23.554348 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:23.586751 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fb45a4-1d4c-4839-b772-dbc3c60556ab None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:23.629895 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8e10c5-9b17-4c8d-a117-b8b38bf540a9 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:23.634629 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b0d431-eaf0-4e6f-a303-3dc91058dfaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:23.635305 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b0d431-eaf0-4e6f-a303-3dc91058dfaf None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:23.635591 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b0d431-eaf0-4e6f-a303-3dc91058dfaf None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:23.635869 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b0d431-eaf0-4e6f-a303-3dc91058dfaf None admin] PATH_INFO: `/v3/endpoints` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:23.636733 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b0d431-eaf0-4e6f-a303-3dc91058dfaf None admin] RBAC: Adding query filter params (interface=public, service_id=3257339b8e804cae8d4389aeb5c3ff07, region_id=RegionOne) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:23.637049 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b0d431-eaf0-4e6f-a303-3dc91058dfaf None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:23.638583 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b0d431-eaf0-4e6f-a303-3dc91058dfaf None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:23.645780 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 459/900] 10.222.0.42 () {56 vars in 1251 bytes} [Thu Jun 16 17:31:23 2022] GET /identity/v3/endpoints?service_id=3257339b8e804cae8d4389aeb5c3ff07&interface=public®ion_id=RegionOne => generated 194 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:23.651812 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b0d431-eaf0-4e6f-a303-3dc91058dfaf None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:23.670779 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb26e29-23de-459a-a3f5-3a944a5ca813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:23.671976 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb26e29-23de-459a-a3f5-3a944a5ca813 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:23.672403 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb26e29-23de-459a-a3f5-3a944a5ca813 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:23.672624 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb26e29-23de-459a-a3f5-3a944a5ca813 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:23.673474 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb26e29-23de-459a-a3f5-3a944a5ca813 None admin] RBAC: Adding query filter params (role.id=6758650940fd4374ab9e18d56545575e, scope.project.id=8abb87f2f141451a8c6ed832db82be18, user.id=71443c30b4554719b5549493bf3cc00c) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:23.673851 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb26e29-23de-459a-a3f5-3a944a5ca813 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:23.675361 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb26e29-23de-459a-a3f5-3a944a5ca813 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:23.683028 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 442/901] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jun 16 17:31:23 2022] GET /identity/v3/role_assignments?user.id=71443c30b4554719b5549493bf3cc00c&scope.project.id=8abb87f2f141451a8c6ed832db82be18&role.id=6758650940fd4374ab9e18d56545575e => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:31:23.783617 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b0d431-eaf0-4e6f-a303-3dc91058dfaf None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:23.823907 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565d70ba-89ae-4116-9177-5e9d34d610a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:23.824635 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565d70ba-89ae-4116-9177-5e9d34d610a2 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:23.824924 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565d70ba-89ae-4116-9177-5e9d34d610a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:23.825206 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565d70ba-89ae-4116-9177-5e9d34d610a2 None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:23.825900 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565d70ba-89ae-4116-9177-5e9d34d610a2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:23.827311 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565d70ba-89ae-4116-9177-5e9d34d610a2 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:23.834921 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 460/902] 10.222.0.42 () {56 vars in 1087 bytes} [Thu Jun 16 17:31:23 2022] GET /identity/v3/services => generated 1268 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 17:31:24.653362 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781ad036-878c-401c-8908-ed6228fe7fc0 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:24.654004 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781ad036-878c-401c-8908-ed6228fe7fc0 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:24.654004 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781ad036-878c-401c-8908-ed6228fe7fc0 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:24.656235 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 443/903] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:31:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:31:24.662071 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edab353-325d-4aeb-a2a2-220437fc5718 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:24.662610 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edab353-325d-4aeb-a2a2-220437fc5718 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:24.663002 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edab353-325d-4aeb-a2a2-220437fc5718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:24.684750 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-5edab353-325d-4aeb-a2a2-220437fc5718 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:31:24.686732 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-5edab353-325d-4aeb-a2a2-220437fc5718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:31:24.690348 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:24.771070 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 461/904] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:31:24 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 17:31:24.795066 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781ad036-878c-401c-8908-ed6228fe7fc0 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:24.818195 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:24.945175 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781ad036-878c-401c-8908-ed6228fe7fc0 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:24.985028 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616c267e-8c47-4227-86a3-53da7cb87470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:24.986247 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616c267e-8c47-4227-86a3-53da7cb87470 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:24.986669 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616c267e-8c47-4227-86a3-53da7cb87470 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:24.987084 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616c267e-8c47-4227-86a3-53da7cb87470 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:24.988086 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616c267e-8c47-4227-86a3-53da7cb87470 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:24.990235 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616c267e-8c47-4227-86a3-53da7cb87470 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:25.009328 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-616c267e-8c47-4227-86a3-53da7cb87470 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 17:31:25.009328 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:25.009328 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 17:31:25.009328 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 17:31:25.009328 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 17:31:25.009328 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 17:31:25.009328 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 17:31:25.009328 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:31:25.009328 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 17:31:25.009328 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:31:25.009328 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:31:25.009328 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:31:25.009328 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:25.009328 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:31:25.009328 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:31:25.009328 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:31:25.010724 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:31:25.010724 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:31:25.010724 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:31:25.010724 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:31:25.010724 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:31:25.010724 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:31:25.010724 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:31:25.010724 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 17:31:25.010724 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 17:31:25.010724 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 17:31:25.010724 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 17:31:25.010724 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:31:25.010724 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:31:25.010724 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:31:25.010724 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:31:25.010724 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:31:25.012081 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:31:25.012081 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:31:25.012081 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:31:25.012081 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:31:25.012081 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:31:25.012081 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:31:25.012081 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:31:25.012081 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:31:25.012081 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:31:25.012081 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:31:25.012081 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:31:25.012081 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 17:31:25.012081 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:31:25.012081 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 17:31:25.012081 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:31:25.012081 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 444/905] 10.222.0.42 () {56 vars in 1132 bytes} [Thu Jun 16 17:31:24 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:31:25.022400 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edab353-325d-4aeb-a2a2-220437fc5718 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:25.143790 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edab353-325d-4aeb-a2a2-220437fc5718 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:25.178847 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e90a67f-dc6c-477c-9345-d63b2613c1e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:25.179540 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e90a67f-dc6c-477c-9345-d63b2613c1e4 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:25.179821 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e90a67f-dc6c-477c-9345-d63b2613c1e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:25.180099 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e90a67f-dc6c-477c-9345-d63b2613c1e4 None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:25.180843 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e90a67f-dc6c-477c-9345-d63b2613c1e4 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:25.181145 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e90a67f-dc6c-477c-9345-d63b2613c1e4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:25.182590 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e90a67f-dc6c-477c-9345-d63b2613c1e4 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:25.188960 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 462/906] 10.222.0.42 () {56 vars in 1126 bytes} [Thu Jun 16 17:31:25 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:25.195035 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616c267e-8c47-4227-86a3-53da7cb87470 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:25.288850 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:25.320396 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616c267e-8c47-4227-86a3-53da7cb87470 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:25.353593 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb69af8a-1b47-4d17-833e-e473c868e3e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:25.354267 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb69af8a-1b47-4d17-833e-e473c868e3e2 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:25.354550 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb69af8a-1b47-4d17-833e-e473c868e3e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:25.354824 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb69af8a-1b47-4d17-833e-e473c868e3e2 None admin] PATH_INFO: `/v3/services` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:25.355600 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb69af8a-1b47-4d17-833e-e473c868e3e2 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:25.355831 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb69af8a-1b47-4d17-833e-e473c868e3e2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:25.357235 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb69af8a-1b47-4d17-833e-e473c868e3e2 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:25.363683 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 445/907] 10.222.0.42 () {56 vars in 1126 bytes} [Thu Jun 16 17:31:25 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:25.369874 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e90a67f-dc6c-477c-9345-d63b2613c1e4 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:25.380712 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:25.487763 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e90a67f-dc6c-477c-9345-d63b2613c1e4 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:25.523788 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ac431c-2451-4b59-bc2d-9d709945a1fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:25.524482 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ac431c-2451-4b59-bc2d-9d709945a1fe None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:25.524790 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ac431c-2451-4b59-bc2d-9d709945a1fe None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:25.525036 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ac431c-2451-4b59-bc2d-9d709945a1fe None admin] PATH_INFO: `/v3/endpoints` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:25.525819 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ac431c-2451-4b59-bc2d-9d709945a1fe None admin] RBAC: Adding query filter params (interface=public, service_id=3d70cd52fb43450f8df8924d9c62b8ae, region_id=RegionOne) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:25.526129 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ac431c-2451-4b59-bc2d-9d709945a1fe None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:25.527492 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ac431c-2451-4b59-bc2d-9d709945a1fe None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:25.533106 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 463/908] 10.222.0.42 () {56 vars in 1251 bytes} [Thu Jun 16 17:31:25 2022] GET /identity/v3/endpoints?service_id=3d70cd52fb43450f8df8924d9c62b8ae&interface=public®ion_id=RegionOne => generated 194 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:25.539175 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb69af8a-1b47-4d17-833e-e473c868e3e2 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:25.649975 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb69af8a-1b47-4d17-833e-e473c868e3e2 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:25.692488 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e313cb-ac78-444e-b8b7-b8f92ebc5861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:25.693260 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e313cb-ac78-444e-b8b7-b8f92ebc5861 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:25.693438 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e313cb-ac78-444e-b8b7-b8f92ebc5861 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:25.693710 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e313cb-ac78-444e-b8b7-b8f92ebc5861 None admin] PATH_INFO: `/v3/services` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:25.694399 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e313cb-ac78-444e-b8b7-b8f92ebc5861 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:25.695812 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e313cb-ac78-444e-b8b7-b8f92ebc5861 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:25.702272 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 446/909] 10.222.0.42 () {56 vars in 1087 bytes} [Thu Jun 16 17:31:25 2022] GET /identity/v3/services => generated 1268 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:31:25.968940 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c0366f-ded8-4ec9-8c83-64c6e737e433 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:25.969485 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c0366f-ded8-4ec9-8c83-64c6e737e433 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:25.969485 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c0366f-ded8-4ec9-8c83-64c6e737e433 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:25.971152 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 464/910] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:31:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:31:25.976419 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95927c10-03f9-443d-bff7-8dd1b47bffc1 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:25.976722 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95927c10-03f9-443d-bff7-8dd1b47bffc1 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:25.977007 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95927c10-03f9-443d-bff7-8dd1b47bffc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:25.983160 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:25.998235 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-95927c10-03f9-443d-bff7-8dd1b47bffc1 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:31:26.000074 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-95927c10-03f9-443d-bff7-8dd1b47bffc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:31:26.083180 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df94d923-3864-4268-ac0c-6590ab391397 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:26.083500 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df94d923-3864-4268-ac0c-6590ab391397 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:26.083743 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df94d923-3864-4268-ac0c-6590ab391397 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:26.085458 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 465/911] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:31:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:31:26.090233 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 447/912] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:31:25 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:31:26.090713 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09a042b-e23c-49a9-8d65-6291f55d823d None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:26.091072 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09a042b-e23c-49a9-8d65-6291f55d823d None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:26.092240 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09a042b-e23c-49a9-8d65-6291f55d823d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:26.112491 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-b09a042b-e23c-49a9-8d65-6291f55d823d None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:31:26.113692 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-b09a042b-e23c-49a9-8d65-6291f55d823d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:31:26.114485 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95927c10-03f9-443d-bff7-8dd1b47bffc1 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:26.194113 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:26.199875 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 466/913] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:31:26 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 17:31:26.219493 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09a042b-e23c-49a9-8d65-6291f55d823d None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:26.238755 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95927c10-03f9-443d-bff7-8dd1b47bffc1 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:26.289753 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8d4b0a-82f5-4fb0-a775-e72a9d8f7f49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:26.291405 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8d4b0a-82f5-4fb0-a775-e72a9d8f7f49 None admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:26.291792 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8d4b0a-82f5-4fb0-a775-e72a9d8f7f49 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:26.292081 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8d4b0a-82f5-4fb0-a775-e72a9d8f7f49 None admin] PATH_INFO: `/v3/services` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:26.292876 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8d4b0a-82f5-4fb0-a775-e72a9d8f7f49 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:26.294250 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8d4b0a-82f5-4fb0-a775-e72a9d8f7f49 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:26.296639 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:26.310778 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 448/914] 10.222.0.42 () {60 vars in 1141 bytes} [Thu Jun 16 17:31:26 2022] POST /identity/v3/services => generated 241 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:31:26.312343 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dacb24-e524-49d2-a7d9-96309c7f387a None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:26.312664 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dacb24-e524-49d2-a7d9-96309c7f387a None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:26.313180 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dacb24-e524-49d2-a7d9-96309c7f387a None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:26.314900 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 449/915] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:31:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:31:26.322042 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e810b4e5-a6c7-47d2-9600-a13dcabd84d6 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:26.322042 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e810b4e5-a6c7-47d2-9600-a13dcabd84d6 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:26.322042 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e810b4e5-a6c7-47d2-9600-a13dcabd84d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:26.342913 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-e810b4e5-a6c7-47d2-9600-a13dcabd84d6 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:31:26.345906 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-e810b4e5-a6c7-47d2-9600-a13dcabd84d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:31:26.372195 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09a042b-e23c-49a9-8d65-6291f55d823d None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:26.431818 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef4c72a-cff7-4f48-9e59-6a09f68fb735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:26.433102 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef4c72a-cff7-4f48-9e59-6a09f68fb735 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:26.433591 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef4c72a-cff7-4f48-9e59-6a09f68fb735 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:26.434205 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef4c72a-cff7-4f48-9e59-6a09f68fb735 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:26.448898 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 450/916] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:31:26 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:31:26.451272 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef4c72a-cff7-4f48-9e59-6a09f68fb735 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:31:26.460514 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-9ef4c72a-cff7-4f48-9e59-6a09f68fb735 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 17:31:26.460514 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:26.460514 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:31:26.460514 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:31:26.460514 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:31:26.460514 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:31:26.460514 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:31:26.460514 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:31:26.460514 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:31:26.460514 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:31:26.460514 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:31:26.460514 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:31:26.460514 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:31:26.460514 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:31:26.460514 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:31:26.460514 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:31:26.461584 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:31:26.461584 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:31:26.461584 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:31:26.461584 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:31:26.461584 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:31:26.461584 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:31:26.461584 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:31:26.461584 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:31:26.461584 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:31:26.461584 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:31:26.461584 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:31:26.461584 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:31:26.461584 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:31:26.461584 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:31:26.461584 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:31:26.461584 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:31:26.462528 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:31:26.462528 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:31:26.462528 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:31:26.462528 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:31:26.462528 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:31:26.462528 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:31:26.462528 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:31:26.462528 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:31:26.462528 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:31:26.462528 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:31:26.462528 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 17:31:26.462528 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:31:26.466052 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 467/917] 10.222.0.42 () {56 vars in 1120 bytes} [Thu Jun 16 17:31:26 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:31:26.472867 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e810b4e5-a6c7-47d2-9600-a13dcabd84d6 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:26.474557 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef4c72a-cff7-4f48-9e59-6a09f68fb735 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:26.552143 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:26.622189 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef4c72a-cff7-4f48-9e59-6a09f68fb735 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:26.628415 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e810b4e5-a6c7-47d2-9600-a13dcabd84d6 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:26.661799 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:26.667220 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa430c6-8e78-4a0a-91a3-698ed98ffa7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:26.668409 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa430c6-8e78-4a0a-91a3-698ed98ffa7f None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:26.668848 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa430c6-8e78-4a0a-91a3-698ed98ffa7f None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:26.669300 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa430c6-8e78-4a0a-91a3-698ed98ffa7f None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:26.670171 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa430c6-8e78-4a0a-91a3-698ed98ffa7f None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:26.672416 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa430c6-8e78-4a0a-91a3-698ed98ffa7f None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:26.673697 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1332d209-dc3d-4afb-b0d5-9586b1092757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:26.675100 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1332d209-dc3d-4afb-b0d5-9586b1092757 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:26.675485 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1332d209-dc3d-4afb-b0d5-9586b1092757 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:26.675863 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1332d209-dc3d-4afb-b0d5-9586b1092757 None admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:26.677170 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1332d209-dc3d-4afb-b0d5-9586b1092757 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:26.677859 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1332d209-dc3d-4afb-b0d5-9586b1092757 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:26.679343 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1332d209-dc3d-4afb-b0d5-9586b1092757 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:26.686518 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-5aa430c6-8e78-4a0a-91a3-698ed98ffa7f None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 17:31:26.686518 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:26.686518 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 17:31:26.686518 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 17:31:26.686518 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 17:31:26.686518 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 17:31:26.686518 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 17:31:26.686518 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:31:26.686518 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 17:31:26.686518 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:31:26.686518 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:31:26.686518 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:31:26.686518 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:26.686518 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:31:26.686518 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:31:26.686518 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:31:26.686518 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:31:26.688150 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:31:26.688150 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:31:26.688150 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:31:26.688150 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:31:26.688150 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:31:26.688150 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:31:26.688150 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 17:31:26.688150 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 17:31:26.688150 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 17:31:26.688150 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 17:31:26.688150 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:31:26.688150 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:31:26.688150 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:31:26.688150 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:31:26.688150 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:31:26.688150 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:31:26.689061 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:31:26.689061 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:31:26.689061 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:31:26.689061 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:31:26.689061 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:31:26.689061 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:31:26.689061 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:31:26.689061 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:31:26.689061 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:31:26.689061 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:31:26.689061 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 17:31:26.689061 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:31:26.689061 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 17:31:26.689061 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:31:26.691943 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 451/919] 10.222.0.42 () {56 vars in 1115 bytes} [Thu Jun 16 17:31:26 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:26.692064 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 468/919] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Jun 16 17:31:26 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:31:26.699737 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1332d209-dc3d-4afb-b0d5-9586b1092757 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:26.701464 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa430c6-8e78-4a0a-91a3-698ed98ffa7f None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:26.840027 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1332d209-dc3d-4afb-b0d5-9586b1092757 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:26.853346 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa430c6-8e78-4a0a-91a3-698ed98ffa7f None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:26.876489 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:26.897849 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356e109e-2040-4608-9893-415596137d60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:26.899313 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356e109e-2040-4608-9893-415596137d60 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:26.899531 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be35e24c-f94d-4f74-a3e2-9837a22d40b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:26.900227 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be35e24c-f94d-4f74-a3e2-9837a22d40b9 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:26.900409 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356e109e-2040-4608-9893-415596137d60 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:26.900542 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be35e24c-f94d-4f74-a3e2-9837a22d40b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:26.900825 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be35e24c-f94d-4f74-a3e2-9837a22d40b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:26.901139 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356e109e-2040-4608-9893-415596137d60 None admin] PATH_INFO: `/v3/services` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:26.902870 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356e109e-2040-4608-9893-415596137d60 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:26.903397 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356e109e-2040-4608-9893-415596137d60 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:26.905161 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356e109e-2040-4608-9893-415596137d60 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:26.913729 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 452/920] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Jun 16 17:31:26 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:26.917612 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be35e24c-f94d-4f74-a3e2-9837a22d40b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:26.919522 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be35e24c-f94d-4f74-a3e2-9837a22d40b9 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:26.919806 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356e109e-2040-4608-9893-415596137d60 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:26.941333 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-be35e24c-f94d-4f74-a3e2-9837a22d40b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:31:26.941333 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:26.941333 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:31:26.941333 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:31:26.941333 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:31:26.941333 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:31:26.941333 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:31:26.941333 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:31:26.941333 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:31:26.941333 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:31:26.941333 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:31:26.941333 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:31:26.941333 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:31:26.941333 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:31:26.941333 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:31:26.947782 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:31:26.947782 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:31:26.947782 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:31:26.947782 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:31:26.947782 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:31:26.947782 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:31:26.947782 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:31:26.947782 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:31:26.947782 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:31:26.947782 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:31:26.947782 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:31:26.947782 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:31:26.947782 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:31:26.947782 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:31:26.947782 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:31:26.947782 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:31:26.947782 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:31:26.948838 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:31:26.948838 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:31:26.948838 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:31:26.948838 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:31:26.950950 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 469/921] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jun 16 17:31:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:31:26.957264 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be35e24c-f94d-4f74-a3e2-9837a22d40b9 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:26.996944 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:27.076493 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356e109e-2040-4608-9893-415596137d60 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:27.105738 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be35e24c-f94d-4f74-a3e2-9837a22d40b9 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:27.120577 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73c8e56-e695-4744-89a8-0f8144c1f10f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:27.123110 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73c8e56-e695-4744-89a8-0f8144c1f10f None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:27.123397 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73c8e56-e695-4744-89a8-0f8144c1f10f None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:27.123678 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73c8e56-e695-4744-89a8-0f8144c1f10f None admin] PATH_INFO: `/v3/services` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:27.125016 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73c8e56-e695-4744-89a8-0f8144c1f10f None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:27.125523 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73c8e56-e695-4744-89a8-0f8144c1f10f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:27.127439 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73c8e56-e695-4744-89a8-0f8144c1f10f None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:27.135141 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 453/922] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Jun 16 17:31:26 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:31:27.142215 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73c8e56-e695-4744-89a8-0f8144c1f10f None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:27.162463 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae9c198-7730-41a0-9e5f-b0666058d394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:27.163973 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae9c198-7730-41a0-9e5f-b0666058d394 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:27.164500 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae9c198-7730-41a0-9e5f-b0666058d394 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:27.164957 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae9c198-7730-41a0-9e5f-b0666058d394 None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:27.165874 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae9c198-7730-41a0-9e5f-b0666058d394 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:27.166967 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae9c198-7730-41a0-9e5f-b0666058d394 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:27.168900 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae9c198-7730-41a0-9e5f-b0666058d394 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:27.195887 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 470/923] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jun 16 17:31:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:31:27.196532 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae9c198-7730-41a0-9e5f-b0666058d394 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:27.245381 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:27.285042 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73c8e56-e695-4744-89a8-0f8144c1f10f None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:27.299874 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:27.325316 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0793e37b-dc46-426e-9ade-bcd577025b33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:27.325978 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0793e37b-dc46-426e-9ade-bcd577025b33 None admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:27.326261 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0793e37b-dc46-426e-9ade-bcd577025b33 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:27.326541 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0793e37b-dc46-426e-9ade-bcd577025b33 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:27.327382 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0793e37b-dc46-426e-9ade-bcd577025b33 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:27.328930 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0793e37b-dc46-426e-9ade-bcd577025b33 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:27.353499 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae9c198-7730-41a0-9e5f-b0666058d394 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:27.356524 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 454/924] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Jun 16 17:31:27 2022] POST /identity/v3/endpoints => generated 346 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:31:27.406505 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ea0f58-746f-4a01-a0c2-b59bbf368228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:27.407745 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ea0f58-746f-4a01-a0c2-b59bbf368228 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:27.408246 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ea0f58-746f-4a01-a0c2-b59bbf368228 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:27.408855 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ea0f58-746f-4a01-a0c2-b59bbf368228 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:27.417499 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ea0f58-746f-4a01-a0c2-b59bbf368228 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:27.419324 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ea0f58-746f-4a01-a0c2-b59bbf368228 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:27.439714 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-89ea0f58-746f-4a01-a0c2-b59bbf368228 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 17:31:27.439714 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:27.439714 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:31:27.439714 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:31:27.439714 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:31:27.439714 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:31:27.439714 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:31:27.439714 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:31:27.439714 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:31:27.439714 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:31:27.439714 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:31:27.439714 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:31:27.439714 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:31:27.439714 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:31:27.439714 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:31:27.439714 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:31:27.440864 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:31:27.440864 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:31:27.440864 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:31:27.440864 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:31:27.440864 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:31:27.440864 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:31:27.440864 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:31:27.440864 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:31:27.440864 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:31:27.440864 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:31:27.440864 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:31:27.440864 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:31:27.440864 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:31:27.440864 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:31:27.440864 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:31:27.440864 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:31:27.440864 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:31:27.440864 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:31:27.441841 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:31:27.441841 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:31:27.441841 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:31:27.441841 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:31:27.441841 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:31:27.441841 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:31:27.441841 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:31:27.441841 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:31:27.441841 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 17:31:27.441841 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:31:27.444807 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 471/925] 10.222.0.42 () {56 vars in 1117 bytes} [Thu Jun 16 17:31:27 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:31:27.450926 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0793e37b-dc46-426e-9ade-bcd577025b33 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:27.573169 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0793e37b-dc46-426e-9ade-bcd577025b33 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:27.609338 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee657d9-fd91-43f5-a18c-2b8ac1ea57fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:27.610377 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee657d9-fd91-43f5-a18c-2b8ac1ea57fd None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:27.610867 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee657d9-fd91-43f5-a18c-2b8ac1ea57fd None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:27.611359 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee657d9-fd91-43f5-a18c-2b8ac1ea57fd None admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:27.612212 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee657d9-fd91-43f5-a18c-2b8ac1ea57fd None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:27.612554 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee657d9-fd91-43f5-a18c-2b8ac1ea57fd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:27.614010 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee657d9-fd91-43f5-a18c-2b8ac1ea57fd None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:27.635195 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 455/926] 10.222.0.42 () {56 vars in 1149 bytes} [Thu Jun 16 17:31:27 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:27.642199 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ea0f58-746f-4a01-a0c2-b59bbf368228 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:27.732059 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:27.748995 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ea0f58-746f-4a01-a0c2-b59bbf368228 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:27.785094 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebce00e-0dd3-445a-b557-b29c3b2bb6c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:27.785889 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebce00e-0dd3-445a-b557-b29c3b2bb6c9 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:27.786173 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebce00e-0dd3-445a-b557-b29c3b2bb6c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:27.786446 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebce00e-0dd3-445a-b557-b29c3b2bb6c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:27.797812 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebce00e-0dd3-445a-b557-b29c3b2bb6c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:27.799265 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebce00e-0dd3-445a-b557-b29c3b2bb6c9 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:27.817754 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-2ebce00e-0dd3-445a-b557-b29c3b2bb6c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:31:27.817754 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:27.817754 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:31:27.817754 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:31:27.817754 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:31:27.817754 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:31:27.817754 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:31:27.817754 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:31:27.817754 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:31:27.817754 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:31:27.817754 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:31:27.817754 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:31:27.817754 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:31:27.817754 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:31:27.817754 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:31:27.818696 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:31:27.818696 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:31:27.818696 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:31:27.818696 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:31:27.818696 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:31:27.818696 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:31:27.818696 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:31:27.818696 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:31:27.818696 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:31:27.818696 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:31:27.818696 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:31:27.818696 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:31:27.818696 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:31:27.818696 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:31:27.818696 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:31:27.818696 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:31:27.818696 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:31:27.819587 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:31:27.819587 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:31:27.819587 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:31:27.819587 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:31:27.819587 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 472/927] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jun 16 17:31:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:31:27.824748 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee657d9-fd91-43f5-a18c-2b8ac1ea57fd None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:27.854084 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:27.932789 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee657d9-fd91-43f5-a18c-2b8ac1ea57fd None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:27.969372 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a89d43-1bf2-4f09-84d8-1db1d6996a80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:27.970046 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a89d43-1bf2-4f09-84d8-1db1d6996a80 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:27.970332 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a89d43-1bf2-4f09-84d8-1db1d6996a80 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:27.970608 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a89d43-1bf2-4f09-84d8-1db1d6996a80 None admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:27.971334 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a89d43-1bf2-4f09-84d8-1db1d6996a80 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:27.971642 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a89d43-1bf2-4f09-84d8-1db1d6996a80 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:27.973042 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a89d43-1bf2-4f09-84d8-1db1d6996a80 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:27.985128 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 456/928] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jun 16 17:31:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:27.990682 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebce00e-0dd3-445a-b557-b29c3b2bb6c9 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:28.094166 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebce00e-0dd3-445a-b557-b29c3b2bb6c9 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:28.130613 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d04bc46-c7c6-44be-9892-037323ca9fb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:28.132242 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d04bc46-c7c6-44be-9892-037323ca9fb1 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:28.132785 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d04bc46-c7c6-44be-9892-037323ca9fb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:28.133246 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d04bc46-c7c6-44be-9892-037323ca9fb1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:28.136993 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:28.142822 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d04bc46-c7c6-44be-9892-037323ca9fb1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:28.144714 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d04bc46-c7c6-44be-9892-037323ca9fb1 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:28.161148 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-9d04bc46-c7c6-44be-9892-037323ca9fb1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:31:28.161148 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:28.161148 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:31:28.161148 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:31:28.161148 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:31:28.161148 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:31:28.161148 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:31:28.161148 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:31:28.161148 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:31:28.161148 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:31:28.161148 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:31:28.161148 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:31:28.161148 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:31:28.161148 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:31:28.161148 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:31:28.162115 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:31:28.162115 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:31:28.162115 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:31:28.162115 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:31:28.162115 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:31:28.162115 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:31:28.162115 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:31:28.162115 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:31:28.162115 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:31:28.162115 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:31:28.162115 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:31:28.162115 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:31:28.162115 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:31:28.162115 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:31:28.162115 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:31:28.162115 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:31:28.162115 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:31:28.163009 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:31:28.163009 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:31:28.163009 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:31:28.163009 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:31:28.163009 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:31:28.163009 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:31:28.163009 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:31:28.163009 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:31:28.165272 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 473/929] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jun 16 17:31:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:31:28.171488 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a89d43-1bf2-4f09-84d8-1db1d6996a80 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:28.200732 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74abc5ff-dcef-4372-b463-1ff28b6a7ad5 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:28.201383 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74abc5ff-dcef-4372-b463-1ff28b6a7ad5 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:28.201960 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74abc5ff-dcef-4372-b463-1ff28b6a7ad5 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:28.204434 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 474/930] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:31:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:31:28.209388 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f304a1-160f-4f9e-be5c-b99678327714 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:28.209751 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f304a1-160f-4f9e-be5c-b99678327714 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:28.210104 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f304a1-160f-4f9e-be5c-b99678327714 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:28.228529 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-09f304a1-160f-4f9e-be5c-b99678327714 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:31:28.229946 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-09f304a1-160f-4f9e-be5c-b99678327714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:31:28.268275 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:28.280891 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a89d43-1bf2-4f09-84d8-1db1d6996a80 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:28.315856 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 475/931] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:31:28 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 17:31:28.324403 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1437d0-e00f-4466-8744-8ea1d9d12b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:28.324403 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1437d0-e00f-4466-8744-8ea1d9d12b23 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:28.324403 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1437d0-e00f-4466-8744-8ea1d9d12b23 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:28.324403 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1437d0-e00f-4466-8744-8ea1d9d12b23 None admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:28.324805 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1437d0-e00f-4466-8744-8ea1d9d12b23 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:28.324805 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1437d0-e00f-4466-8744-8ea1d9d12b23 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:28.324805 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1437d0-e00f-4466-8744-8ea1d9d12b23 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:28.340996 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f304a1-160f-4f9e-be5c-b99678327714 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:28.348690 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 457/932] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Jun 16 17:31:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:28.349796 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1437d0-e00f-4466-8744-8ea1d9d12b23 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:28.459532 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f304a1-160f-4f9e-be5c-b99678327714 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:28.491523 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:28.498707 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1437d0-e00f-4466-8744-8ea1d9d12b23 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:28.514451 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf910195-3936-4d2c-9985-70cb52e33af6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:28.516497 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf910195-3936-4d2c-9985-70cb52e33af6 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:28.517234 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf910195-3936-4d2c-9985-70cb52e33af6 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:28.518526 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf910195-3936-4d2c-9985-70cb52e33af6 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:28.519859 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf910195-3936-4d2c-9985-70cb52e33af6 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:28.523489 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf910195-3936-4d2c-9985-70cb52e33af6 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:28.547595 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-cf910195-3936-4d2c-9985-70cb52e33af6 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 17:31:28.547595 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:28.547595 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 17:31:28.547595 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 17:31:28.547595 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 17:31:28.547595 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 17:31:28.547595 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 17:31:28.547595 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:31:28.547595 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 17:31:28.547595 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:31:28.547595 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:31:28.547595 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:31:28.547595 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:28.547595 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:31:28.547595 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:31:28.547595 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:31:28.548778 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:31:28.548778 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:31:28.548778 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:31:28.548778 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:31:28.548778 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:31:28.548778 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:31:28.548778 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:31:28.548778 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 17:31:28.548778 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 17:31:28.548778 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 17:31:28.548778 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 17:31:28.548778 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:31:28.548778 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:31:28.548778 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:31:28.548778 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:31:28.548778 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:31:28.549618 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:31:28.549618 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:31:28.549618 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:31:28.549618 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:31:28.549618 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:31:28.549618 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:31:28.549618 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:31:28.549618 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:31:28.549618 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:31:28.549618 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:31:28.549618 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:31:28.549618 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 17:31:28.549618 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:31:28.549618 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 17:31:28.549618 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:31:28.553824 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 476/933] 10.222.0.42 () {56 vars in 1132 bytes} [Thu Jun 16 17:31:28 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:31:28.557215 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636f425d-c18f-40cd-874f-7f0bc5e6fd6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:28.558555 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf910195-3936-4d2c-9985-70cb52e33af6 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:28.558954 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636f425d-c18f-40cd-874f-7f0bc5e6fd6c None admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:28.559553 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636f425d-c18f-40cd-874f-7f0bc5e6fd6c None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:28.560068 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636f425d-c18f-40cd-874f-7f0bc5e6fd6c None admin] PATH_INFO: `/v3/projects/8abb87f2f141451a8c6ed832db82be18/users/71443c30b4554719b5549493bf3cc00c/roles/6758650940fd4374ab9e18d56545575e` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:28.563426 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636f425d-c18f-40cd-874f-7f0bc5e6fd6c None admin] RBAC: Authorizing `identity:create_grant(project_id=8abb87f2f141451a8c6ed832db82be18, user_id=71443c30b4554719b5549493bf3cc00c, role_id=6758650940fd4374ab9e18d56545575e)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:28.565207 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636f425d-c18f-40cd-874f-7f0bc5e6fd6c None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:28.583272 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 458/934] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Jun 16 17:31:28 2022] PUT /identity/v3/projects/8abb87f2f141451a8c6ed832db82be18/users/71443c30b4554719b5549493bf3cc00c/roles/6758650940fd4374ab9e18d56545575e => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:31:28.683494 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf910195-3936-4d2c-9985-70cb52e33af6 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:28.725575 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e6ecd8-b7fe-4957-951e-d7e87c799002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:28.726400 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e6ecd8-b7fe-4957-951e-d7e87c799002 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:28.726692 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e6ecd8-b7fe-4957-951e-d7e87c799002 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:28.726975 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e6ecd8-b7fe-4957-951e-d7e87c799002 None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:28.727800 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e6ecd8-b7fe-4957-951e-d7e87c799002 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:28.728042 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e6ecd8-b7fe-4957-951e-d7e87c799002 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:28.729546 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e6ecd8-b7fe-4957-951e-d7e87c799002 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:28.740224 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 477/935] 10.222.0.42 () {56 vars in 1126 bytes} [Thu Jun 16 17:31:28 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:31:28.742782 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636f425d-c18f-40cd-874f-7f0bc5e6fd6c None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:28.756020 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:28.786723 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca03b2fa-26e9-4fe8-9323-6cba1f1ed754 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:28.787359 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca03b2fa-26e9-4fe8-9323-6cba1f1ed754 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:28.787847 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca03b2fa-26e9-4fe8-9323-6cba1f1ed754 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:28.789772 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 478/936] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:31:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:31:28.795436 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6ef581-af2d-46c3-92e5-128e15955127 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:28.795728 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6ef581-af2d-46c3-92e5-128e15955127 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:28.796008 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6ef581-af2d-46c3-92e5-128e15955127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:28.815530 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-bb6ef581-af2d-46c3-92e5-128e15955127 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:31:28.816745 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-bb6ef581-af2d-46c3-92e5-128e15955127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:31:28.867216 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636f425d-c18f-40cd-874f-7f0bc5e6fd6c None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:28.888993 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:28.911167 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 479/937] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:31:28 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:31:28.919737 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ff9ca7-ba8b-426b-8071-8c8aa380be21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:28.920703 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ff9ca7-ba8b-426b-8071-8c8aa380be21 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:28.921763 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ff9ca7-ba8b-426b-8071-8c8aa380be21 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:28.922265 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ff9ca7-ba8b-426b-8071-8c8aa380be21 None admin] PATH_INFO: `/v3/services` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:28.923175 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ff9ca7-ba8b-426b-8071-8c8aa380be21 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:28.923694 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ff9ca7-ba8b-426b-8071-8c8aa380be21 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:28.925290 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ff9ca7-ba8b-426b-8071-8c8aa380be21 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:28.934070 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 459/938] 10.222.0.42 () {56 vars in 1126 bytes} [Thu Jun 16 17:31:28 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:28.935424 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6ef581-af2d-46c3-92e5-128e15955127 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:28.939748 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ff9ca7-ba8b-426b-8071-8c8aa380be21 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:29.026539 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:29.050443 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6ef581-af2d-46c3-92e5-128e15955127 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:29.063172 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ff9ca7-ba8b-426b-8071-8c8aa380be21 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:29.090213 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb9062c-9564-40a0-b35a-81afc155e5ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:29.091111 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb9062c-9564-40a0-b35a-81afc155e5ce None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:29.091460 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb9062c-9564-40a0-b35a-81afc155e5ce None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:29.091808 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb9062c-9564-40a0-b35a-81afc155e5ce None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:29.092553 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb9062c-9564-40a0-b35a-81afc155e5ce None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:29.094708 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb9062c-9564-40a0-b35a-81afc155e5ce None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:29.106271 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39bc99e-c92e-40ca-ae41-4aef687d40ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:29.107150 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39bc99e-c92e-40ca-ae41-4aef687d40ed None admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:29.107616 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39bc99e-c92e-40ca-ae41-4aef687d40ed None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:29.107792 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-2fb9062c-9564-40a0-b35a-81afc155e5ce None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 17:31:29.107792 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:29.107792 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 17:31:29.107792 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 17:31:29.107792 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 17:31:29.107792 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 17:31:29.107792 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 17:31:29.107792 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:31:29.107792 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 17:31:29.107792 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:31:29.107792 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:31:29.107792 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:31:29.107792 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:29.107792 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:31:29.107792 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:31:29.107792 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:31:29.107792 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:31:29.108853 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:31:29.108853 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:31:29.108853 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:31:29.108853 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:31:29.108853 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:31:29.108853 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:31:29.108853 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 17:31:29.108853 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 17:31:29.108853 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 17:31:29.108853 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 17:31:29.108853 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:31:29.108853 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:31:29.108853 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:31:29.108853 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:31:29.108853 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:31:29.108853 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:31:29.110194 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:31:29.110194 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:31:29.110194 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:31:29.110194 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:31:29.110194 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:31:29.110194 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:31:29.110194 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:31:29.110194 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:31:29.110194 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:31:29.110194 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:31:29.110194 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 17:31:29.110194 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:31:29.110194 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 17:31:29.110194 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:31:29.111764 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39bc99e-c92e-40ca-ae41-4aef687d40ed None admin] PATH_INFO: `/v3/endpoints` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:29.112764 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39bc99e-c92e-40ca-ae41-4aef687d40ed None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:29.114389 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39bc99e-c92e-40ca-ae41-4aef687d40ed None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:29.114959 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 480/939] 10.222.0.42 () {56 vars in 1126 bytes} [Thu Jun 16 17:31:28 2022] GET /identity/v3/services/object-store => generated 93 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:31:29.115976 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb9062c-9564-40a0-b35a-81afc155e5ce None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:29.141173 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 460/940] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Jun 16 17:31:28 2022] POST /identity/v3/endpoints => generated 347 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:31:29.186327 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:29.250004 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb9062c-9564-40a0-b35a-81afc155e5ce None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:29.307494 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5af270-bc60-4932-837a-43c0db4f67e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:29.307494 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5af270-bc60-4932-837a-43c0db4f67e0 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:29.307494 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5af270-bc60-4932-837a-43c0db4f67e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:29.307494 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5af270-bc60-4932-837a-43c0db4f67e0 None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:29.308387 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5af270-bc60-4932-837a-43c0db4f67e0 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:29.308387 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5af270-bc60-4932-837a-43c0db4f67e0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:29.308387 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5af270-bc60-4932-837a-43c0db4f67e0 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:29.308597 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 481/941] 10.222.0.42 () {56 vars in 1122 bytes} [Thu Jun 16 17:31:29 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:29.317526 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39bc99e-c92e-40ca-ae41-4aef687d40ed None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:29.448005 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39bc99e-c92e-40ca-ae41-4aef687d40ed None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:29.477152 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:29.487182 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f542f89c-3add-4fb6-97fc-b47bac1b2385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:29.488277 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f542f89c-3add-4fb6-97fc-b47bac1b2385 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:29.488789 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f542f89c-3add-4fb6-97fc-b47bac1b2385 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:29.489266 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f542f89c-3add-4fb6-97fc-b47bac1b2385 None admin] PATH_INFO: `/v3/services` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:29.490160 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f542f89c-3add-4fb6-97fc-b47bac1b2385 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:29.490682 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f542f89c-3add-4fb6-97fc-b47bac1b2385 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:29.492255 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f542f89c-3add-4fb6-97fc-b47bac1b2385 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:29.498106 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 461/942] 10.222.0.42 () {56 vars in 1122 bytes} [Thu Jun 16 17:31:29 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:29.504832 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5af270-bc60-4932-837a-43c0db4f67e0 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:29.612078 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5af270-bc60-4932-837a-43c0db4f67e0 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:29.621355 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:29.648600 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08028d23-6310-4adc-aa9f-fce30368d515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:29.649267 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08028d23-6310-4adc-aa9f-fce30368d515 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:29.649553 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08028d23-6310-4adc-aa9f-fce30368d515 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:29.649834 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08028d23-6310-4adc-aa9f-fce30368d515 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:29.650622 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08028d23-6310-4adc-aa9f-fce30368d515 None admin] RBAC: Adding query filter params (interface=public, service_id=4dc1989961e7418b8d24d94ac42def2b, region_id=RegionOne) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:29.650938 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08028d23-6310-4adc-aa9f-fce30368d515 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:29.652308 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08028d23-6310-4adc-aa9f-fce30368d515 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:29.658583 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 482/943] 10.222.0.42 () {56 vars in 1251 bytes} [Thu Jun 16 17:31:29 2022] GET /identity/v3/endpoints?service_id=4dc1989961e7418b8d24d94ac42def2b&interface=public®ion_id=RegionOne => generated 194 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:29.663987 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f542f89c-3add-4fb6-97fc-b47bac1b2385 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:29.749200 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786c3170-3a9e-4244-a316-aea3ccd236b4 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:29.749987 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786c3170-3a9e-4244-a316-aea3ccd236b4 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:29.750227 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786c3170-3a9e-4244-a316-aea3ccd236b4 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:29.752546 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 483/944] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:31:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:31:29.757363 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcea218-e4fe-455c-a99f-32ec84641281 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:29.758023 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcea218-e4fe-455c-a99f-32ec84641281 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:29.758407 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcea218-e4fe-455c-a99f-32ec84641281 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:29.780154 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f542f89c-3add-4fb6-97fc-b47bac1b2385 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:29.782268 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-8fcea218-e4fe-455c-a99f-32ec84641281 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:31:29.784805 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-8fcea218-e4fe-455c-a99f-32ec84641281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:31:29.819994 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd986336-923c-4960-8b54-8e6c9b80ab25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:29.821076 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd986336-923c-4960-8b54-8e6c9b80ab25 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:29.821583 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd986336-923c-4960-8b54-8e6c9b80ab25 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:29.822057 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd986336-923c-4960-8b54-8e6c9b80ab25 None admin] PATH_INFO: `/v3/services` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:29.822924 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd986336-923c-4960-8b54-8e6c9b80ab25 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:29.824566 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd986336-923c-4960-8b54-8e6c9b80ab25 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:29.836415 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 462/945] 10.222.0.42 () {56 vars in 1087 bytes} [Thu Jun 16 17:31:29 2022] GET /identity/v3/services => generated 1497 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:31:29.885741 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 484/946] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:31:29 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:31:29.903830 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd986336-923c-4960-8b54-8e6c9b80ab25 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:29.979671 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:30.032157 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd986336-923c-4960-8b54-8e6c9b80ab25 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:30.070277 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565d7a51-2c0b-479c-b5ff-9b38fd914518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:30.070947 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565d7a51-2c0b-479c-b5ff-9b38fd914518 None admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:30.071237 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565d7a51-2c0b-479c-b5ff-9b38fd914518 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:30.071521 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565d7a51-2c0b-479c-b5ff-9b38fd914518 None admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:30.072219 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565d7a51-2c0b-479c-b5ff-9b38fd914518 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:30.073686 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565d7a51-2c0b-479c-b5ff-9b38fd914518 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:30.075066 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: WARNING keystone.server.flask.common [None req-565d7a51-2c0b-479c-b5ff-9b38fd914518 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 16 17:31:30.096090 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.sql.core [None req-565d7a51-2c0b-479c-b5ff-9b38fd914518 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 16 17:31:30.096090 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 16 17:31:30.096090 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [parameters: {'id': 'f1b67abc26a64c3cae56aa09baed94a1', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 16 17:31:30.096090 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80357) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 16 17:31:30.114176 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-565d7a51-2c0b-479c-b5ff-9b38fd914518 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id f1b67abc26a64c3cae56aa09baed94a1.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id f1b67abc26a64c3cae56aa09baed94a1. Jun 16 17:31:30.114176 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:30.114176 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 16 17:31:30.114176 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 16 17:31:30.114176 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 16 17:31:30.114176 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 16 17:31:30.114176 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 16 17:31:30.114176 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application result = self._query(query) Jun 16 17:31:30.114176 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 16 17:31:30.114176 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application conn.query(q) Jun 16 17:31:30.114176 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 16 17:31:30.114176 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 16 17:31:30.114176 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 16 17:31:30.115029 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application result.read() Jun 16 17:31:30.115029 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 16 17:31:30.115029 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 16 17:31:30.115029 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 16 17:31:30.115029 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 16 17:31:30.115029 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 16 17:31:30.115029 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 16 17:31:30.115029 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 16 17:31:30.115029 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 16 17:31:30.115029 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 16 17:31:30.115029 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:31:30.115029 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 16 17:31:30.115029 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:31:30.115029 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:30.115029 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 16 17:31:30.115029 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 16 17:31:30.115029 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jun 16 17:31:30.115029 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return project_ref.to_dict() Jun 16 17:31:30.115970 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 16 17:31:30.115970 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application next(self.gen) Jun 16 17:31:30.115970 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 16 17:31:30.115970 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application yield resource Jun 16 17:31:30.115970 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 16 17:31:30.115970 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application next(self.gen) Jun 16 17:31:30.115970 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 16 17:31:30.115970 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application self.session.rollback() Jun 16 17:31:30.115970 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 16 17:31:30.115970 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application self.force_reraise() Jun 16 17:31:30.115970 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 16 17:31:30.115970 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise self.value Jun 16 17:31:30.115970 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 16 17:31:30.115970 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 16 17:31:30.115970 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 16 17:31:30.115970 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application session.commit() Jun 16 17:31:30.115970 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 16 17:31:30.116892 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 16 17:31:30.116892 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 16 17:31:30.116892 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application self._prepare_impl() Jun 16 17:31:30.116892 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 16 17:31:30.116892 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application self.session.flush() Jun 16 17:31:30.116892 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 16 17:31:30.116892 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application self._flush(objects) Jun 16 17:31:30.116892 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 16 17:31:30.116892 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 16 17:31:30.116892 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 16 17:31:30.116892 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application compat.raise_( Jun 16 17:31:30.116892 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 16 17:31:30.116892 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception Jun 16 17:31:30.116892 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 16 17:31:30.116892 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application flush_context.execute() Jun 16 17:31:30.116892 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 16 17:31:30.116892 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rec.execute(self) Jun 16 17:31:30.117798 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 16 17:31:30.117798 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 16 17:31:30.117798 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 16 17:31:30.117798 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 16 17:31:30.117798 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 16 17:31:30.117798 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 16 17:31:30.117798 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 16 17:31:30.117798 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 16 17:31:30.117798 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 16 17:31:30.117798 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 16 17:31:30.117798 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 16 17:31:30.117798 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 16 17:31:30.117798 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 16 17:31:30.117798 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 16 17:31:30.117798 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 16 17:31:30.118575 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 16 17:31:30.118575 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 16 17:31:30.118575 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception Jun 16 17:31:30.118575 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 16 17:31:30.118575 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 16 17:31:30.118575 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 16 17:31:30.118575 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 16 17:31:30.118575 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 16 17:31:30.118575 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application result = self._query(query) Jun 16 17:31:30.118575 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 16 17:31:30.118575 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application conn.query(q) Jun 16 17:31:30.118575 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 16 17:31:30.118575 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 16 17:31:30.118575 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 16 17:31:30.118575 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application result.read() Jun 16 17:31:30.118575 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 16 17:31:30.118575 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 16 17:31:30.119451 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 16 17:31:30.119451 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 16 17:31:30.119451 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 16 17:31:30.119451 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 16 17:31:30.119451 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 16 17:31:30.119451 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 16 17:31:30.119451 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 16 17:31:30.119451 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 16 17:31:30.119451 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [parameters: {'id': 'f1b67abc26a64c3cae56aa09baed94a1', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 16 17:31:30.119451 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 16 17:31:30.119451 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:31:30.119451 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:31:30.119451 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:31:30.119451 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:30.119451 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jun 16 17:31:30.120242 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jun 16 17:31:30.120242 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 16 17:31:30.120242 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 16 17:31:30.120242 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jun 16 17:31:30.120242 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:31:30.120242 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:31:30.120242 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:31:30.120242 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:30.120242 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:31:30.120242 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:31:30.120242 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:31:30.120242 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:31:30.120242 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:31:30.120242 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:31:30.120242 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:31:30.120242 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:31:30.120242 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:31:30.123459 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:31:30.123459 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jun 16 17:31:30.123459 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jun 16 17:31:30.123459 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:31:30.123459 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:31:30.123459 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jun 16 17:31:30.123459 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.Conflict( Jun 16 17:31:30.123459 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id f1b67abc26a64c3cae56aa09baed94a1. Jun 16 17:31:30.123459 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:31:30.123459 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 463/947] 10.222.0.42 () {60 vars in 1140 bytes} [Thu Jun 16 17:31:29 2022] POST /identity/v3/projects => generated 262 bytes in 213 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:31:30.125492 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcea218-e4fe-455c-a99f-32ec84641281 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:30.239193 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcea218-e4fe-455c-a99f-32ec84641281 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:30.240815 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:30.277421 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e4bff9-ca27-4ceb-8e7c-540e575e874b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:30.278234 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e4bff9-ca27-4ceb-8e7c-540e575e874b None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:30.278520 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e4bff9-ca27-4ceb-8e7c-540e575e874b None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:30.278797 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e4bff9-ca27-4ceb-8e7c-540e575e874b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:30.286997 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e4bff9-ca27-4ceb-8e7c-540e575e874b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:30.288402 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e4bff9-ca27-4ceb-8e7c-540e575e874b None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:30.306004 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-a4e4bff9-ca27-4ceb-8e7c-540e575e874b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:31:30.306004 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:30.306004 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:31:30.306004 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:31:30.306004 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:31:30.306004 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:31:30.306004 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:31:30.306004 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:31:30.306004 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:31:30.306004 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:31:30.306004 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:31:30.306004 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:31:30.306004 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:31:30.306004 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:31:30.306004 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:31:30.306922 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:31:30.306922 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:31:30.306922 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:31:30.306922 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:31:30.306922 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:31:30.306922 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:31:30.306922 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:31:30.306922 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:31:30.306922 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:31:30.306922 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:31:30.306922 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:31:30.306922 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:31:30.306922 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:31:30.306922 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:31:30.306922 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:31:30.306922 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:31:30.306922 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:31:30.307805 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:31:30.307805 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:31:30.307805 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:31:30.307805 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:31:30.307805 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:31:30.307805 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:31:30.307805 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:31:30.307805 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:31:30.307805 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 485/948] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jun 16 17:31:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:31:30.313407 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565d7a51-2c0b-479c-b5ff-9b38fd914518 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:30.425975 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565d7a51-2c0b-479c-b5ff-9b38fd914518 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:30.441510 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:30.467963 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79be8478-c495-462e-8bf6-fe5f279809cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:30.469439 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79be8478-c495-462e-8bf6-fe5f279809cd None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:30.469551 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79be8478-c495-462e-8bf6-fe5f279809cd None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:30.469813 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79be8478-c495-462e-8bf6-fe5f279809cd None admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:30.470575 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79be8478-c495-462e-8bf6-fe5f279809cd None admin] RBAC: Adding query filter params (name=service) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:30.470887 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79be8478-c495-462e-8bf6-fe5f279809cd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:30.472646 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79be8478-c495-462e-8bf6-fe5f279809cd None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:30.494805 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 464/949] 10.222.0.42 () {56 vars in 1112 bytes} [Thu Jun 16 17:31:30 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:31:30.973563 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90787612-2ed5-4602-8b5b-caad619d1eda None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:30.973889 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90787612-2ed5-4602-8b5b-caad619d1eda None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:30.974135 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90787612-2ed5-4602-8b5b-caad619d1eda None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:30.975767 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 486/950] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:31:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:31:30.981579 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e09fe06-1c0b-4e65-b203-9d29fb4d22fc None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:30.982014 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e09fe06-1c0b-4e65-b203-9d29fb4d22fc None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:30.982179 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e09fe06-1c0b-4e65-b203-9d29fb4d22fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:31.000752 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-1e09fe06-1c0b-4e65-b203-9d29fb4d22fc None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:31:31.002455 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-1e09fe06-1c0b-4e65-b203-9d29fb4d22fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:31:31.080665 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 465/951] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:31:30 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:31:31.101291 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90787612-2ed5-4602-8b5b-caad619d1eda None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:31.209901 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90787612-2ed5-4602-8b5b-caad619d1eda None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:31.246073 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28300d7-6103-490c-9600-2f2712ffe79e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:31.246891 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28300d7-6103-490c-9600-2f2712ffe79e None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:31.247179 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28300d7-6103-490c-9600-2f2712ffe79e None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:31.247459 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28300d7-6103-490c-9600-2f2712ffe79e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:31.259000 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28300d7-6103-490c-9600-2f2712ffe79e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:31:31.268248 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-c28300d7-6103-490c-9600-2f2712ffe79e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 17:31:31.268248 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:31.268248 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:31:31.268248 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:31:31.268248 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:31:31.268248 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:31:31.268248 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:31:31.268248 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:31:31.268248 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:31:31.268248 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:31:31.268248 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:31:31.268248 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:31:31.268248 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:31:31.268248 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:31:31.268248 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:31:31.268248 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:31:31.269208 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:31:31.269208 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:31:31.269208 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:31:31.269208 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:31:31.269208 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:31:31.269208 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:31:31.269208 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:31:31.269208 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:31:31.269208 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:31:31.269208 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:31:31.269208 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:31:31.269208 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:31:31.269208 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:31:31.269208 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:31:31.269208 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:31:31.269208 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:31:31.270215 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:31:31.270215 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:31:31.270215 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:31:31.270215 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:31:31.270215 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:31:31.270215 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:31:31.270215 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:31:31.270215 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:31:31.270215 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:31:31.270215 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:31:31.270215 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 17:31:31.270215 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:31:31.274421 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 487/952] 10.222.0.42 () {56 vars in 1120 bytes} [Thu Jun 16 17:31:31 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:31:31.275782 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e09fe06-1c0b-4e65-b203-9d29fb4d22fc None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:31.381590 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e09fe06-1c0b-4e65-b203-9d29fb4d22fc None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:31.383150 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:31.417666 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9965f7-e987-4d5c-86d0-194b45deb200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:31.418345 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9965f7-e987-4d5c-86d0-194b45deb200 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:31.418630 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9965f7-e987-4d5c-86d0-194b45deb200 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:31.418910 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9965f7-e987-4d5c-86d0-194b45deb200 None admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:31.419664 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9965f7-e987-4d5c-86d0-194b45deb200 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:31.419974 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9965f7-e987-4d5c-86d0-194b45deb200 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:31.421438 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9965f7-e987-4d5c-86d0-194b45deb200 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:31.435274 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 466/953] 10.222.0.42 () {56 vars in 1115 bytes} [Thu Jun 16 17:31:31 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:31.438427 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28300d7-6103-490c-9600-2f2712ffe79e None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:31.469095 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:31.503955 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11df4e9-7388-416a-b914-a84c443e15a8 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:31.504322 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11df4e9-7388-416a-b914-a84c443e15a8 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:31.504562 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11df4e9-7388-416a-b914-a84c443e15a8 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:31.506064 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 467/954] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:31:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:31:31.510963 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff782843-4c5e-40f4-85a8-162938ada61d None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:31.511251 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff782843-4c5e-40f4-85a8-162938ada61d None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:31.511537 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff782843-4c5e-40f4-85a8-162938ada61d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:31.533644 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-ff782843-4c5e-40f4-85a8-162938ada61d None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:31:31.534893 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-ff782843-4c5e-40f4-85a8-162938ada61d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:31:31.555128 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28300d7-6103-490c-9600-2f2712ffe79e None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:31.603896 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea18395-6cf9-4a60-8aff-e6de35fb85a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:31.605136 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea18395-6cf9-4a60-8aff-e6de35fb85a3 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:31.605423 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea18395-6cf9-4a60-8aff-e6de35fb85a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:31.605706 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea18395-6cf9-4a60-8aff-e6de35fb85a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:31.620702 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 468/955] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:31:31 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 17:31:31.620872 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea18395-6cf9-4a60-8aff-e6de35fb85a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:31.622304 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea18395-6cf9-4a60-8aff-e6de35fb85a3 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:31.639400 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff782843-4c5e-40f4-85a8-162938ada61d None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:31.642281 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-cea18395-6cf9-4a60-8aff-e6de35fb85a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:31:31.642281 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:31.642281 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:31:31.642281 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:31:31.642281 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:31:31.642281 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:31:31.642281 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:31:31.642281 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:31:31.642281 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:31:31.642281 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:31:31.642281 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:31:31.642281 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:31:31.642281 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:31:31.642281 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:31:31.642281 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:31:31.643396 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:31:31.643396 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:31:31.643396 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:31:31.643396 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:31:31.643396 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:31:31.643396 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:31:31.643396 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:31:31.643396 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:31:31.643396 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:31:31.643396 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:31:31.643396 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:31:31.643396 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:31:31.643396 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:31:31.643396 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:31:31.643396 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:31:31.643396 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:31:31.643396 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:31:31.644457 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:31:31.644457 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:31:31.644457 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:31:31.644457 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:31:31.648422 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 488/956] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jun 16 17:31:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:31:31.650715 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea18395-6cf9-4a60-8aff-e6de35fb85a3 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:31.754964 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:31.761381 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:31.764958 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff782843-4c5e-40f4-85a8-162938ada61d None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:31.793020 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea18395-6cf9-4a60-8aff-e6de35fb85a3 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:31.819102 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78618fb-c93a-40bf-9067-f1d624f4ebe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:31.820357 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78618fb-c93a-40bf-9067-f1d624f4ebe2 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:31.820929 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78618fb-c93a-40bf-9067-f1d624f4ebe2 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:31.821519 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78618fb-c93a-40bf-9067-f1d624f4ebe2 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:31.822407 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78618fb-c93a-40bf-9067-f1d624f4ebe2 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:31.824124 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78618fb-c93a-40bf-9067-f1d624f4ebe2 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:31.833923 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f9df95-e93b-4b82-8ba8-6bb53eb1cb15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:31.834596 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f9df95-e93b-4b82-8ba8-6bb53eb1cb15 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:31.834882 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f9df95-e93b-4b82-8ba8-6bb53eb1cb15 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:31.835165 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f9df95-e93b-4b82-8ba8-6bb53eb1cb15 None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:31.836127 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f9df95-e93b-4b82-8ba8-6bb53eb1cb15 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:31.836470 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f9df95-e93b-4b82-8ba8-6bb53eb1cb15 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:31.837840 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f9df95-e93b-4b82-8ba8-6bb53eb1cb15 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:31.839468 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-f78618fb-c93a-40bf-9067-f1d624f4ebe2 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 17:31:31.839468 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:31.839468 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 17:31:31.839468 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 17:31:31.839468 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 17:31:31.839468 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 17:31:31.839468 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 17:31:31.839468 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:31:31.839468 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 17:31:31.839468 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:31:31.839468 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:31:31.839468 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:31:31.839468 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:31.839468 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:31:31.839468 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:31:31.839468 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:31:31.839468 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:31:31.840905 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:31:31.840905 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:31:31.840905 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:31:31.840905 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:31:31.840905 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:31:31.840905 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:31:31.840905 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 17:31:31.840905 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 17:31:31.840905 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 17:31:31.840905 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 17:31:31.840905 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:31:31.840905 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:31:31.840905 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:31:31.840905 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:31:31.840905 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:31:31.840905 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:31:31.841795 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:31:31.841795 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:31:31.841795 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:31:31.841795 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:31:31.841795 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:31:31.841795 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:31:31.841795 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:31:31.841795 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:31:31.841795 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:31:31.841795 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:31:31.841795 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 17:31:31.841795 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:31:31.841795 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 17:31:31.841795 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:31:31.849701 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 469/957] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jun 16 17:31:31 2022] GET /identity/v3/services/compute => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:31:31.850862 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78618fb-c93a-40bf-9067-f1d624f4ebe2 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:31.855660 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 489/958] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jun 16 17:31:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:31.857998 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f9df95-e93b-4b82-8ba8-6bb53eb1cb15 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:31.976492 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78618fb-c93a-40bf-9067-f1d624f4ebe2 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:32.009778 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f9df95-e93b-4b82-8ba8-6bb53eb1cb15 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:32.013141 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c267108-bcec-4e1d-82d1-143c98d1857b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:32.013600 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c267108-bcec-4e1d-82d1-143c98d1857b None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:32.013826 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c267108-bcec-4e1d-82d1-143c98d1857b None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:32.014107 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c267108-bcec-4e1d-82d1-143c98d1857b None admin] PATH_INFO: `/v3/services` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:32.014812 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c267108-bcec-4e1d-82d1-143c98d1857b None admin] RBAC: Adding query filter params (name=compute) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:32.015156 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c267108-bcec-4e1d-82d1-143c98d1857b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:32.016975 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c267108-bcec-4e1d-82d1-143c98d1857b None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:32.028132 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 470/959] 10.222.0.42 () {56 vars in 1112 bytes} [Thu Jun 16 17:31:31 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:32.030482 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c267108-bcec-4e1d-82d1-143c98d1857b None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:32.061188 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f80bbc2-7eb5-4393-a718-7ac6f2138f96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:32.062565 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f80bbc2-7eb5-4393-a718-7ac6f2138f96 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:32.062985 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f80bbc2-7eb5-4393-a718-7ac6f2138f96 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:32.063383 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f80bbc2-7eb5-4393-a718-7ac6f2138f96 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:32.065073 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:32.065941 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:32.073378 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f80bbc2-7eb5-4393-a718-7ac6f2138f96 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:32.075548 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f80bbc2-7eb5-4393-a718-7ac6f2138f96 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:32.092292 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-4f80bbc2-7eb5-4393-a718-7ac6f2138f96 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 17:31:32.092292 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:32.092292 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:31:32.092292 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:31:32.092292 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:31:32.092292 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:31:32.092292 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:31:32.092292 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:31:32.092292 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:31:32.092292 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:31:32.092292 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:31:32.092292 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:31:32.092292 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:31:32.092292 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:31:32.092292 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:31:32.092292 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:31:32.094291 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:31:32.094291 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:31:32.094291 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:31:32.094291 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:31:32.094291 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:31:32.094291 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:31:32.094291 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:31:32.094291 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:31:32.094291 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:31:32.094291 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:31:32.094291 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:31:32.094291 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:31:32.094291 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:31:32.094291 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:31:32.094291 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:31:32.094291 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:31:32.094291 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:31:32.094291 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:31:32.095513 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:31:32.095513 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:31:32.095513 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:31:32.095513 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:31:32.095513 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:31:32.095513 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:31:32.095513 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:31:32.095513 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:31:32.095513 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 17:31:32.095513 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:31:32.098351 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 490/960] 10.222.0.42 () {56 vars in 1117 bytes} [Thu Jun 16 17:31:31 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:31:32.105410 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f80bbc2-7eb5-4393-a718-7ac6f2138f96 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:32.172359 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c267108-bcec-4e1d-82d1-143c98d1857b None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:32.214444 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9351c3f-5f5d-4f4b-80eb-4177f1b5057a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:32.215120 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9351c3f-5f5d-4f4b-80eb-4177f1b5057a None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:32.215406 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9351c3f-5f5d-4f4b-80eb-4177f1b5057a None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:32.215703 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9351c3f-5f5d-4f4b-80eb-4177f1b5057a None admin] PATH_INFO: `/v3/services` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:32.216413 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9351c3f-5f5d-4f4b-80eb-4177f1b5057a None admin] RBAC: Adding query filter params (type=compute) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:32.216733 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9351c3f-5f5d-4f4b-80eb-4177f1b5057a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:32.218186 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9351c3f-5f5d-4f4b-80eb-4177f1b5057a None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:32.223667 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 471/961] 10.222.0.42 () {56 vars in 1112 bytes} [Thu Jun 16 17:31:32 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:32.225327 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96439a69-e711-447b-8501-57ecb0a5c7f7 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:32.225618 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96439a69-e711-447b-8501-57ecb0a5c7f7 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:32.225901 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96439a69-e711-447b-8501-57ecb0a5c7f7 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:32.227103 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f80bbc2-7eb5-4393-a718-7ac6f2138f96 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:32.231588 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 472/962] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:31:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:31:32.233597 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a96c51-8552-4592-b9c8-c5bf42aecac1 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:32.233922 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a96c51-8552-4592-b9c8-c5bf42aecac1 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:32.234313 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a96c51-8552-4592-b9c8-c5bf42aecac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:32.257613 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-e0a96c51-8552-4592-b9c8-c5bf42aecac1 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:31:32.259451 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-e0a96c51-8552-4592-b9c8-c5bf42aecac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:31:32.271188 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a425d6-9056-476c-894d-120397aaf619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:32.272596 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a425d6-9056-476c-894d-120397aaf619 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:32.273173 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a425d6-9056-476c-894d-120397aaf619 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:32.273672 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a425d6-9056-476c-894d-120397aaf619 None admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:32.274816 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a425d6-9056-476c-894d-120397aaf619 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:32.275337 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a425d6-9056-476c-894d-120397aaf619 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:32.277173 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a425d6-9056-476c-894d-120397aaf619 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:32.299058 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 491/963] 10.222.0.42 () {56 vars in 1149 bytes} [Thu Jun 16 17:31:32 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:31:32.305516 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a425d6-9056-476c-894d-120397aaf619 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:32.364418 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 473/964] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:31:32 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:31:32.383692 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:32.386107 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a96c51-8552-4592-b9c8-c5bf42aecac1 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:32.412319 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:32.458485 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a425d6-9056-476c-894d-120397aaf619 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:32.501614 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee97621-d765-4011-acbb-e9bacb7a95fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:32.502373 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee97621-d765-4011-acbb-e9bacb7a95fe None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:32.502661 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee97621-d765-4011-acbb-e9bacb7a95fe None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:32.503040 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee97621-d765-4011-acbb-e9bacb7a95fe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:32.519088 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a96c51-8552-4592-b9c8-c5bf42aecac1 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:32.556626 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a086fa87-c979-4007-b562-98b481bfb19e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:32.557640 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a086fa87-c979-4007-b562-98b481bfb19e None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:32.557640 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a086fa87-c979-4007-b562-98b481bfb19e None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:32.557948 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a086fa87-c979-4007-b562-98b481bfb19e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:32.558619 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a086fa87-c979-4007-b562-98b481bfb19e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:32.559914 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a086fa87-c979-4007-b562-98b481bfb19e None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:32.572115 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-a086fa87-c979-4007-b562-98b481bfb19e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 17:31:32.572115 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:32.572115 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 17:31:32.572115 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 17:31:32.572115 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 17:31:32.572115 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 17:31:32.572115 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 17:31:32.572115 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:31:32.572115 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 17:31:32.572115 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:31:32.572115 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:31:32.572115 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:31:32.572115 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:32.572115 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:31:32.572115 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:31:32.572115 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:31:32.572115 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:31:32.573162 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:31:32.573162 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:31:32.573162 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:31:32.573162 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:31:32.573162 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:31:32.573162 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:31:32.573162 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 17:31:32.573162 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 17:31:32.573162 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 17:31:32.573162 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 17:31:32.573162 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:31:32.573162 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:31:32.573162 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:31:32.573162 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:31:32.573162 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:31:32.573162 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:31:32.574067 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:31:32.574067 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:31:32.574067 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:31:32.574067 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:31:32.574067 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:31:32.574067 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:31:32.574067 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:31:32.574067 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:31:32.574067 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:31:32.574067 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:31:32.574067 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 17:31:32.574067 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:31:32.574067 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 17:31:32.574067 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:31:32.574067 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 474/965] 10.222.0.42 () {56 vars in 1126 bytes} [Thu Jun 16 17:31:32 2022] GET /identity/v3/services/object-store => generated 93 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:31:32.579783 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a086fa87-c979-4007-b562-98b481bfb19e None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:32.586863 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee97621-d765-4011-acbb-e9bacb7a95fe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:32.588470 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee97621-d765-4011-acbb-e9bacb7a95fe None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:32.647966 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:32.692060 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:32.715850 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a086fa87-c979-4007-b562-98b481bfb19e None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:32.722915 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 492/966] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jun 16 17:31:32 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 16 17:31:32.724559 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee97621-d765-4011-acbb-e9bacb7a95fe None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:32.760796 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3018cd3-f5a9-4e5d-ada4-d9ee46983b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:32.761472 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3018cd3-f5a9-4e5d-ada4-d9ee46983b0d None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:32.761821 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3018cd3-f5a9-4e5d-ada4-d9ee46983b0d None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:32.762121 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3018cd3-f5a9-4e5d-ada4-d9ee46983b0d None admin] PATH_INFO: `/v3/services` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:32.762729 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3018cd3-f5a9-4e5d-ada4-d9ee46983b0d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:32.763651 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3018cd3-f5a9-4e5d-ada4-d9ee46983b0d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:32.764477 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3018cd3-f5a9-4e5d-ada4-d9ee46983b0d None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:32.774627 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 475/967] 10.222.0.42 () {56 vars in 1122 bytes} [Thu Jun 16 17:31:32 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:31:32.774627 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dccb56-8009-4092-8762-5fb22e9252ef None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:32.774627 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dccb56-8009-4092-8762-5fb22e9252ef None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:32.774627 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dccb56-8009-4092-8762-5fb22e9252ef None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:32.774627 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 476/968] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:31:32 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:31:32.777369 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dccb56-8009-4092-8762-5fb22e9252ef None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:32.844530 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee97621-d765-4011-acbb-e9bacb7a95fe None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:32.888633 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dccb56-8009-4092-8762-5fb22e9252ef None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:32.892683 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bf9e9d-8eff-411d-9d1c-23c4841c8323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:32.893191 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bf9e9d-8eff-411d-9d1c-23c4841c8323 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:32.893472 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bf9e9d-8eff-411d-9d1c-23c4841c8323 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:32.893748 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bf9e9d-8eff-411d-9d1c-23c4841c8323 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:32.904474 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bf9e9d-8eff-411d-9d1c-23c4841c8323 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:32.906488 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bf9e9d-8eff-411d-9d1c-23c4841c8323 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:32.927672 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-f0bf9e9d-8eff-411d-9d1c-23c4841c8323 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:31:32.927672 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:32.927672 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:31:32.927672 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:31:32.927672 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:31:32.927672 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:31:32.927672 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:31:32.927672 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:31:32.927672 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:31:32.927672 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:31:32.927672 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:31:32.927672 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:31:32.927672 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:31:32.927672 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:31:32.927672 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:31:32.928724 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:31:32.928724 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:31:32.928724 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:31:32.928724 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:31:32.928724 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:31:32.928724 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:31:32.928724 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:31:32.928724 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:31:32.928724 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:31:32.928724 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:31:32.928724 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:31:32.928724 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:31:32.928724 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:31:32.928724 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:31:32.928724 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:31:32.928724 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:31:32.928724 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:31:32.929669 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:31:32.929669 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:31:32.929669 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:31:32.929669 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:31:32.929669 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 493/969] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jun 16 17:31:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:31:32.930521 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d89341-e85e-4e87-b2fc-2f21b081fc6b None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:32.930521 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d89341-e85e-4e87-b2fc-2f21b081fc6b None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:32.930681 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d89341-e85e-4e87-b2fc-2f21b081fc6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:32.938072 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c35f39c-b144-4f34-964d-7db227808138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:32.938318 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:32.938754 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c35f39c-b144-4f34-964d-7db227808138 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:32.939180 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c35f39c-b144-4f34-964d-7db227808138 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:32.939336 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c35f39c-b144-4f34-964d-7db227808138 None admin] PATH_INFO: `/v3/services` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:32.940130 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c35f39c-b144-4f34-964d-7db227808138 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:32.940680 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c35f39c-b144-4f34-964d-7db227808138 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:32.942259 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c35f39c-b144-4f34-964d-7db227808138 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:32.947767 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 477/970] 10.222.0.42 () {56 vars in 1122 bytes} [Thu Jun 16 17:31:32 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:32.949468 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c35f39c-b144-4f34-964d-7db227808138 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:32.950370 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-e4d89341-e85e-4e87-b2fc-2f21b081fc6b None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:31:32.951554 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-e4d89341-e85e-4e87-b2fc-2f21b081fc6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:31:32.965197 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:33.027661 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 494/971] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:31:32 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:31:33.028983 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d89341-e85e-4e87-b2fc-2f21b081fc6b None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:33.067999 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c35f39c-b144-4f34-964d-7db227808138 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:33.105015 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a32798-3a04-4fba-ad27-1e2f27ad2892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:33.105600 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a32798-3a04-4fba-ad27-1e2f27ad2892 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:33.105884 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a32798-3a04-4fba-ad27-1e2f27ad2892 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:33.106158 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a32798-3a04-4fba-ad27-1e2f27ad2892 None admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:33.106874 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a32798-3a04-4fba-ad27-1e2f27ad2892 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:33.107179 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a32798-3a04-4fba-ad27-1e2f27ad2892 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:33.108564 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a32798-3a04-4fba-ad27-1e2f27ad2892 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:33.125772 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 478/972] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jun 16 17:31:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:33.125772 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a32798-3a04-4fba-ad27-1e2f27ad2892 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:33.156997 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d89341-e85e-4e87-b2fc-2f21b081fc6b None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:33.191202 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:33.201578 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4573a3c7-3480-49b0-b63a-6ac69b960235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:33.202257 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4573a3c7-3480-49b0-b63a-6ac69b960235 None admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:33.202710 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4573a3c7-3480-49b0-b63a-6ac69b960235 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:33.203002 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4573a3c7-3480-49b0-b63a-6ac69b960235 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:33.203739 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4573a3c7-3480-49b0-b63a-6ac69b960235 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:33.205137 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4573a3c7-3480-49b0-b63a-6ac69b960235 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:33.208559 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:33.227004 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 495/973] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Jun 16 17:31:33 2022] POST /identity/v3/endpoints => generated 349 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:31:33.228212 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4573a3c7-3480-49b0-b63a-6ac69b960235 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:33.238217 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a32798-3a04-4fba-ad27-1e2f27ad2892 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:33.291223 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231fd060-688f-4997-96d2-538ce586e861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:33.292540 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231fd060-688f-4997-96d2-538ce586e861 None admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:33.292855 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231fd060-688f-4997-96d2-538ce586e861 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:33.293139 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231fd060-688f-4997-96d2-538ce586e861 None admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:33.293876 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231fd060-688f-4997-96d2-538ce586e861 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:33.295377 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231fd060-688f-4997-96d2-538ce586e861 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:33.310049 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.sql.core [None req-231fd060-688f-4997-96d2-538ce586e861 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 16 17:31:33.310049 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 16 17:31:33.310049 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [parameters: {'user_id': 'e94b957084d3457280b5fe9082e5482c', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 16 17:31:33.310049 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80357) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 16 17:31:33.318931 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-231fd060-688f-4997-96d2-538ce586e861 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 16 17:31:33.318931 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:33.318931 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 16 17:31:33.318931 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 16 17:31:33.318931 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 16 17:31:33.318931 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 16 17:31:33.318931 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 16 17:31:33.318931 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application result = self._query(query) Jun 16 17:31:33.318931 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 16 17:31:33.318931 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application conn.query(q) Jun 16 17:31:33.318931 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 16 17:31:33.318931 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 16 17:31:33.318931 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 16 17:31:33.318931 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application result.read() Jun 16 17:31:33.318931 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 16 17:31:33.319894 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 16 17:31:33.319894 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 16 17:31:33.319894 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 16 17:31:33.319894 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 16 17:31:33.319894 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 16 17:31:33.319894 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 16 17:31:33.319894 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 16 17:31:33.319894 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 16 17:31:33.319894 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:31:33.319894 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 16 17:31:33.319894 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:31:33.319894 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:33.319894 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 16 17:31:33.319894 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 16 17:31:33.319894 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jun 16 17:31:33.319894 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jun 16 17:31:33.319894 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 16 17:31:33.319894 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application next(self.gen) Jun 16 17:31:33.320867 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 16 17:31:33.320867 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application yield resource Jun 16 17:31:33.320867 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 16 17:31:33.320867 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application next(self.gen) Jun 16 17:31:33.320867 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 16 17:31:33.320867 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application self.session.rollback() Jun 16 17:31:33.320867 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 16 17:31:33.320867 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application self.force_reraise() Jun 16 17:31:33.320867 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 16 17:31:33.320867 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise self.value Jun 16 17:31:33.320867 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 16 17:31:33.320867 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 16 17:31:33.320867 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 16 17:31:33.320867 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application session.commit() Jun 16 17:31:33.320867 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 16 17:31:33.320867 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 16 17:31:33.320867 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 16 17:31:33.321853 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application self._prepare_impl() Jun 16 17:31:33.321853 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 16 17:31:33.321853 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application self.session.flush() Jun 16 17:31:33.321853 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 16 17:31:33.321853 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application self._flush(objects) Jun 16 17:31:33.321853 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 16 17:31:33.321853 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 16 17:31:33.321853 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 16 17:31:33.321853 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application compat.raise_( Jun 16 17:31:33.321853 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 16 17:31:33.321853 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception Jun 16 17:31:33.321853 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 16 17:31:33.321853 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application flush_context.execute() Jun 16 17:31:33.321853 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 16 17:31:33.321853 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rec.execute(self) Jun 16 17:31:33.321853 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 16 17:31:33.321853 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 16 17:31:33.322762 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 16 17:31:33.322762 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 16 17:31:33.322762 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jun 16 17:31:33.322762 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application result = connection._execute_20( Jun 16 17:31:33.322762 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 16 17:31:33.322762 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 16 17:31:33.322762 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 16 17:31:33.322762 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 16 17:31:33.322762 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 16 17:31:33.322762 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 16 17:31:33.322762 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 16 17:31:33.322762 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 16 17:31:33.322762 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 16 17:31:33.322762 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 16 17:31:33.322762 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 16 17:31:33.322762 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception Jun 16 17:31:33.323670 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 16 17:31:33.323670 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 16 17:31:33.323670 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 16 17:31:33.323670 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 16 17:31:33.323670 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 16 17:31:33.323670 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application result = self._query(query) Jun 16 17:31:33.323670 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 16 17:31:33.323670 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application conn.query(q) Jun 16 17:31:33.323670 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 16 17:31:33.323670 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 16 17:31:33.323670 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 16 17:31:33.323670 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application result.read() Jun 16 17:31:33.323670 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 16 17:31:33.323670 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 16 17:31:33.323670 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 16 17:31:33.323670 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 16 17:31:33.324858 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 16 17:31:33.324858 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 16 17:31:33.324858 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 16 17:31:33.324858 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 16 17:31:33.324858 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 16 17:31:33.324858 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 16 17:31:33.324858 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [parameters: {'user_id': 'e94b957084d3457280b5fe9082e5482c', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 16 17:31:33.324858 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 16 17:31:33.324858 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:31:33.324858 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:31:33.324858 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:31:33.324858 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:33.324858 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:31:33.324858 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:31:33.324858 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:31:33.325853 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:31:33.325853 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:31:33.325853 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:31:33.325853 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:31:33.325853 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:31:33.325853 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:31:33.325853 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:31:33.325853 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jun 16 17:31:33.325853 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jun 16 17:31:33.325853 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:31:33.325853 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:31:33.325853 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:31:33.325853 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:31:33.325853 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:31:33.325853 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:31:33.325853 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jun 16 17:31:33.325853 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jun 16 17:31:33.328240 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jun 16 17:31:33.328240 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jun 16 17:31:33.328240 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 16 17:31:33.328240 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 16 17:31:33.328240 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 16 17:31:33.328240 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:31:33.328240 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 479/974] 10.222.0.42 () {60 vars in 1131 bytes} [Thu Jun 16 17:31:33 2022] POST /identity/v3/users => generated 160 bytes in 199 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:31:33.330292 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231fd060-688f-4997-96d2-538ce586e861 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:33.378649 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4573a3c7-3480-49b0-b63a-6ac69b960235 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:33.418689 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe695775-d2f6-4344-9d88-c015163bdf42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:33.419787 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe695775-d2f6-4344-9d88-c015163bdf42 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:33.420265 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe695775-d2f6-4344-9d88-c015163bdf42 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:33.420739 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe695775-d2f6-4344-9d88-c015163bdf42 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:33.428511 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe695775-d2f6-4344-9d88-c015163bdf42 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:33.430992 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe695775-d2f6-4344-9d88-c015163bdf42 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:33.448360 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-fe695775-d2f6-4344-9d88-c015163bdf42 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:31:33.448360 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:33.448360 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:31:33.448360 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:31:33.448360 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:31:33.448360 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:31:33.448360 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:31:33.448360 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:31:33.448360 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:31:33.448360 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:31:33.448360 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:31:33.448360 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:31:33.448360 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:31:33.448360 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:31:33.448360 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:31:33.449546 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:31:33.449546 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:31:33.449546 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:31:33.449546 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:31:33.449546 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:31:33.449546 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:31:33.449546 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:31:33.449546 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:31:33.449546 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:31:33.449546 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:31:33.449546 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:31:33.449546 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:31:33.449546 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:31:33.449546 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:31:33.449546 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:31:33.449546 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:31:33.449546 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:31:33.450522 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:31:33.450522 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:31:33.450522 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:31:33.450522 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:31:33.450522 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:31:33.450522 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:31:33.450522 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:31:33.450522 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:31:33.450522 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 496/975] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jun 16 17:31:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:31:33.458907 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe695775-d2f6-4344-9d88-c015163bdf42 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:33.472560 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231fd060-688f-4997-96d2-538ce586e861 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:33.506040 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:33.518556 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaf67b3-0b0c-43e3-857b-6c0a5611798b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:33.519463 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaf67b3-0b0c-43e3-857b-6c0a5611798b None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:33.519810 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaf67b3-0b0c-43e3-857b-6c0a5611798b None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:33.520154 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaf67b3-0b0c-43e3-857b-6c0a5611798b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:33.522641 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:33.530382 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caaf67b3-0b0c-43e3-857b-6c0a5611798b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:33.533752 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caaf67b3-0b0c-43e3-857b-6c0a5611798b None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:33.550225 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-caaf67b3-0b0c-43e3-857b-6c0a5611798b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 17:31:33.550225 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:33.550225 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:31:33.550225 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:31:33.550225 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:31:33.550225 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:31:33.550225 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:31:33.550225 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:31:33.550225 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:31:33.550225 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:31:33.550225 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:31:33.550225 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:31:33.550225 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:31:33.550225 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:31:33.550225 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:31:33.550225 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:31:33.551214 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:31:33.551214 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:31:33.551214 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:31:33.551214 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:31:33.551214 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:31:33.551214 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:31:33.551214 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:31:33.551214 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:31:33.551214 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:31:33.551214 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:31:33.551214 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:31:33.551214 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:31:33.551214 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:31:33.551214 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:31:33.551214 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:31:33.551214 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:31:33.551214 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:31:33.551214 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:31:33.552158 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:31:33.552158 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:31:33.552158 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:31:33.552158 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:31:33.552158 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:31:33.552158 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:31:33.552158 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:31:33.552158 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:31:33.552158 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 17:31:33.552158 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:31:33.552158 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 480/976] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jun 16 17:31:33 2022] GET /identity/v3/users/cinder => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:31:33.557467 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaf67b3-0b0c-43e3-857b-6c0a5611798b None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:33.578466 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe695775-d2f6-4344-9d88-c015163bdf42 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:33.615838 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5373f40d-972f-49e5-910c-dcd6d4f7042a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:33.616535 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5373f40d-972f-49e5-910c-dcd6d4f7042a None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:33.616818 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5373f40d-972f-49e5-910c-dcd6d4f7042a None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:33.617093 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5373f40d-972f-49e5-910c-dcd6d4f7042a None admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:33.617983 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5373f40d-972f-49e5-910c-dcd6d4f7042a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:33.618392 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5373f40d-972f-49e5-910c-dcd6d4f7042a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:33.619828 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5373f40d-972f-49e5-910c-dcd6d4f7042a None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:33.638089 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 497/977] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Jun 16 17:31:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:31:33.639256 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5373f40d-972f-49e5-910c-dcd6d4f7042a None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:33.665690 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaf67b3-0b0c-43e3-857b-6c0a5611798b None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:33.703091 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a234550f-c217-4430-b834-ebf8747c4fc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:33.703893 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a234550f-c217-4430-b834-ebf8747c4fc4 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:33.704183 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a234550f-c217-4430-b834-ebf8747c4fc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:33.704487 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a234550f-c217-4430-b834-ebf8747c4fc4 None admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:33.705289 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a234550f-c217-4430-b834-ebf8747c4fc4 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:33.705599 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a234550f-c217-4430-b834-ebf8747c4fc4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:33.707238 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a234550f-c217-4430-b834-ebf8747c4fc4 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:33.746057 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5373f40d-972f-49e5-910c-dcd6d4f7042a None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:33.756496 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 481/978] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Jun 16 17:31:33 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:31:33.757468 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:33.785827 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4475987e-b984-404a-9857-e6922c794e98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:33.786501 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4475987e-b984-404a-9857-e6922c794e98 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:33.786779 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4475987e-b984-404a-9857-e6922c794e98 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:33.787052 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4475987e-b984-404a-9857-e6922c794e98 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:33.787892 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4475987e-b984-404a-9857-e6922c794e98 None admin] RBAC: Adding query filter params (role.id=6758650940fd4374ab9e18d56545575e, scope.project.id=8abb87f2f141451a8c6ed832db82be18, user.id=71443c30b4554719b5549493bf3cc00c) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:33.788201 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4475987e-b984-404a-9857-e6922c794e98 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:33.789698 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4475987e-b984-404a-9857-e6922c794e98 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:33.800456 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 498/979] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jun 16 17:31:33 2022] GET /identity/v3/role_assignments?user.id=71443c30b4554719b5549493bf3cc00c&scope.project.id=8abb87f2f141451a8c6ed832db82be18&role.id=6758650940fd4374ab9e18d56545575e => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:31:34.495958 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc50fe9a-ff30-4a38-8c8d-4cd25794466a None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:34.496525 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc50fe9a-ff30-4a38-8c8d-4cd25794466a None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:34.496525 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc50fe9a-ff30-4a38-8c8d-4cd25794466a None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:34.498055 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 482/980] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:31:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:31:34.504341 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31cbb78-04c5-423d-b668-9ad5aee2774e None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:34.504819 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31cbb78-04c5-423d-b668-9ad5aee2774e None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:34.505000 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31cbb78-04c5-423d-b668-9ad5aee2774e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:34.522761 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-b31cbb78-04c5-423d-b668-9ad5aee2774e None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:31:34.523945 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-b31cbb78-04c5-423d-b668-9ad5aee2774e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:31:34.604405 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 499/981] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:31:34 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 17:31:34.622932 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc50fe9a-ff30-4a38-8c8d-4cd25794466a None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:34.676151 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:34.732680 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc50fe9a-ff30-4a38-8c8d-4cd25794466a None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:34.767814 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0254a5ba-a976-4e7b-9fd7-1f5b861aac81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:34.768432 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0254a5ba-a976-4e7b-9fd7-1f5b861aac81 None admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:34.768723 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0254a5ba-a976-4e7b-9fd7-1f5b861aac81 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:34.768998 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0254a5ba-a976-4e7b-9fd7-1f5b861aac81 None admin] PATH_INFO: `/v3/services` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:34.769706 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0254a5ba-a976-4e7b-9fd7-1f5b861aac81 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:34.771063 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0254a5ba-a976-4e7b-9fd7-1f5b861aac81 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:34.785300 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 483/982] 10.222.0.42 () {60 vars in 1141 bytes} [Thu Jun 16 17:31:34 2022] POST /identity/v3/services => generated 242 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:31:35.511882 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d06e8f-4e88-405b-afae-10cbe0d56910 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:35.512429 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d06e8f-4e88-405b-afae-10cbe0d56910 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:35.512512 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d06e8f-4e88-405b-afae-10cbe0d56910 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:35.513976 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 500/983] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:31:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:31:35.519849 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59af7d8b-f29f-4261-b379-134affe2798d None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:35.520316 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59af7d8b-f29f-4261-b379-134affe2798d None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:35.520421 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59af7d8b-f29f-4261-b379-134affe2798d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:35.537837 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-59af7d8b-f29f-4261-b379-134affe2798d None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:31:35.539324 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-59af7d8b-f29f-4261-b379-134affe2798d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:31:35.615749 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 484/984] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:31:35 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:31:35.637981 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d06e8f-4e88-405b-afae-10cbe0d56910 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:35.739533 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d06e8f-4e88-405b-afae-10cbe0d56910 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:35.741913 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:35.774780 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b18a1d4-55d5-4276-99a5-e5db9fb4f502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:35.775507 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b18a1d4-55d5-4276-99a5-e5db9fb4f502 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:35.775787 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b18a1d4-55d5-4276-99a5-e5db9fb4f502 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:35.776059 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b18a1d4-55d5-4276-99a5-e5db9fb4f502 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:35.776723 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b18a1d4-55d5-4276-99a5-e5db9fb4f502 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:35.778075 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b18a1d4-55d5-4276-99a5-e5db9fb4f502 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:35.790381 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-5b18a1d4-55d5-4276-99a5-e5db9fb4f502 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 17:31:35.790381 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:35.790381 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 17:31:35.790381 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 17:31:35.790381 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 17:31:35.790381 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 17:31:35.790381 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 17:31:35.790381 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:31:35.790381 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 17:31:35.790381 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:31:35.790381 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:31:35.790381 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:31:35.790381 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:35.790381 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:31:35.790381 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:31:35.790381 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:31:35.790381 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:31:35.791469 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:31:35.791469 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:31:35.791469 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:31:35.791469 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:31:35.791469 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:31:35.791469 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:31:35.791469 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 17:31:35.791469 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 17:31:35.791469 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 17:31:35.791469 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 17:31:35.791469 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:31:35.791469 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:31:35.791469 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:31:35.791469 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:31:35.791469 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:31:35.791469 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:31:35.792317 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:31:35.792317 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:31:35.792317 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:31:35.792317 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:31:35.792317 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:31:35.792317 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:31:35.792317 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:31:35.792317 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:31:35.792317 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:31:35.792317 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:31:35.792317 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 17:31:35.792317 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:31:35.792317 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 17:31:35.792317 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:31:35.796867 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 501/985] 10.222.0.42 () {56 vars in 1126 bytes} [Thu Jun 16 17:31:35 2022] GET /identity/v3/services/object-store => generated 93 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:31:35.798136 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59af7d8b-f29f-4261-b379-134affe2798d None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:35.829104 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:35.911564 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59af7d8b-f29f-4261-b379-134affe2798d None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:35.949885 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1cc260-7405-4757-9e57-61534151e520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:35.950575 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1cc260-7405-4757-9e57-61534151e520 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:35.950855 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1cc260-7405-4757-9e57-61534151e520 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:35.951127 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1cc260-7405-4757-9e57-61534151e520 None admin] PATH_INFO: `/v3/services` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:35.951853 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1cc260-7405-4757-9e57-61534151e520 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:35.952157 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1cc260-7405-4757-9e57-61534151e520 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:35.954416 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1cc260-7405-4757-9e57-61534151e520 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:35.965232 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 485/986] 10.222.0.42 () {56 vars in 1122 bytes} [Thu Jun 16 17:31:35 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:35.966260 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b18a1d4-55d5-4276-99a5-e5db9fb4f502 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:36.008345 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab503259-74c2-49ae-87b3-29e335275013 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:36.008671 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab503259-74c2-49ae-87b3-29e335275013 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:36.008951 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab503259-74c2-49ae-87b3-29e335275013 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:36.010324 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 486/987] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:31:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:31:36.015951 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44815c11-a4e9-4c6f-8a37-d3852218ae5d None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:36.016249 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44815c11-a4e9-4c6f-8a37-d3852218ae5d None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:36.016631 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44815c11-a4e9-4c6f-8a37-d3852218ae5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:36.033385 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-44815c11-a4e9-4c6f-8a37-d3852218ae5d None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:31:36.034486 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-44815c11-a4e9-4c6f-8a37-d3852218ae5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:31:36.073786 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b18a1d4-55d5-4276-99a5-e5db9fb4f502 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:36.105399 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 487/988] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:31:36 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 17:31:36.107981 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecd5b0d-c3a5-4378-9729-41c1774bc3ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:36.108915 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecd5b0d-c3a5-4378-9729-41c1774bc3ac None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:36.109511 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecd5b0d-c3a5-4378-9729-41c1774bc3ac None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:36.109933 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecd5b0d-c3a5-4378-9729-41c1774bc3ac None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:36.110791 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecd5b0d-c3a5-4378-9729-41c1774bc3ac None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:36.111239 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecd5b0d-c3a5-4378-9729-41c1774bc3ac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:36.112813 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecd5b0d-c3a5-4378-9729-41c1774bc3ac None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:36.117873 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 502/989] 10.222.0.42 () {56 vars in 1122 bytes} [Thu Jun 16 17:31:35 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:31:36.123585 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44815c11-a4e9-4c6f-8a37-d3852218ae5d None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:36.130164 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecd5b0d-c3a5-4378-9729-41c1774bc3ac None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:36.149376 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:36.164050 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:36.227641 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44815c11-a4e9-4c6f-8a37-d3852218ae5d None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:36.238958 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecd5b0d-c3a5-4378-9729-41c1774bc3ac None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:36.263086 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bb045d-987b-4a1c-b871-a711c1e8c584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:36.263850 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bb045d-987b-4a1c-b871-a711c1e8c584 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:36.264129 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bb045d-987b-4a1c-b871-a711c1e8c584 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:36.264543 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bb045d-987b-4a1c-b871-a711c1e8c584 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:36.266139 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bb045d-987b-4a1c-b871-a711c1e8c584 None admin] RBAC: Adding query filter params (interface=admin, service_id=4dc1989961e7418b8d24d94ac42def2b, region_id=RegionOne) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:36.266503 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bb045d-987b-4a1c-b871-a711c1e8c584 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:36.267968 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bb045d-987b-4a1c-b871-a711c1e8c584 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:36.277644 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906f47a0-8349-48fe-9dc5-9cdcf00e0a19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:36.277807 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 488/990] 10.222.0.42 () {56 vars in 1249 bytes} [Thu Jun 16 17:31:36 2022] GET /identity/v3/endpoints?service_id=4dc1989961e7418b8d24d94ac42def2b&interface=admin®ion_id=RegionOne => generated 193 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:36.277910 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906f47a0-8349-48fe-9dc5-9cdcf00e0a19 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:36.277910 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906f47a0-8349-48fe-9dc5-9cdcf00e0a19 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:36.277910 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906f47a0-8349-48fe-9dc5-9cdcf00e0a19 None admin] PATH_INFO: `/v3/services/image` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:36.277910 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906f47a0-8349-48fe-9dc5-9cdcf00e0a19 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:36.277910 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906f47a0-8349-48fe-9dc5-9cdcf00e0a19 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:36.278668 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bb045d-987b-4a1c-b871-a711c1e8c584 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:36.289915 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-906f47a0-8349-48fe-9dc5-9cdcf00e0a19 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 17:31:36.289915 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:36.289915 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 17:31:36.289915 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 17:31:36.289915 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 17:31:36.289915 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 17:31:36.289915 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 17:31:36.289915 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:31:36.289915 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 17:31:36.289915 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:31:36.289915 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:31:36.289915 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:31:36.289915 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:36.289915 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:31:36.289915 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:31:36.289915 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:31:36.289915 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:31:36.290909 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:31:36.290909 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:31:36.290909 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:31:36.290909 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:31:36.290909 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:31:36.290909 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:31:36.290909 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 17:31:36.290909 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 17:31:36.290909 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 17:31:36.290909 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 17:31:36.290909 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:31:36.290909 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:31:36.290909 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:31:36.290909 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:31:36.290909 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:31:36.290909 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:31:36.291767 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:31:36.291767 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:31:36.291767 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:31:36.291767 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:31:36.291767 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:31:36.291767 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:31:36.291767 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:31:36.291767 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:31:36.291767 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:31:36.291767 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:31:36.291767 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 17:31:36.291767 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:31:36.291767 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 17:31:36.291767 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:31:36.291767 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 503/991] 10.222.0.42 () {56 vars in 1105 bytes} [Thu Jun 16 17:31:36 2022] GET /identity/v3/services/image => generated 86 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:31:36.297439 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906f47a0-8349-48fe-9dc5-9cdcf00e0a19 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:36.374763 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:36.388003 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bb045d-987b-4a1c-b871-a711c1e8c584 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:36.400829 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:36.408938 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906f47a0-8349-48fe-9dc5-9cdcf00e0a19 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:36.429809 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44df94f1-6d8a-4d7e-830e-3cacbaad93a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:36.430618 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44df94f1-6d8a-4d7e-830e-3cacbaad93a7 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:36.430909 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44df94f1-6d8a-4d7e-830e-3cacbaad93a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:36.431186 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44df94f1-6d8a-4d7e-830e-3cacbaad93a7 None admin] PATH_INFO: `/v3/services` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:36.431866 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44df94f1-6d8a-4d7e-830e-3cacbaad93a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:36.433284 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44df94f1-6d8a-4d7e-830e-3cacbaad93a7 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:36.440939 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 489/992] 10.222.0.42 () {56 vars in 1087 bytes} [Thu Jun 16 17:31:36 2022] GET /identity/v3/services => generated 1727 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:31:36.462420 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb55c9e-078b-4fd8-98b1-608d0cfc7920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:36.463122 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb55c9e-078b-4fd8-98b1-608d0cfc7920 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:36.463408 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb55c9e-078b-4fd8-98b1-608d0cfc7920 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:36.463687 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb55c9e-078b-4fd8-98b1-608d0cfc7920 None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:36.464449 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb55c9e-078b-4fd8-98b1-608d0cfc7920 None admin] RBAC: Adding query filter params (name=image) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:36.464767 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb55c9e-078b-4fd8-98b1-608d0cfc7920 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:36.466192 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb55c9e-078b-4fd8-98b1-608d0cfc7920 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:36.473442 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 504/993] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jun 16 17:31:36 2022] GET /identity/v3/services?name=image => generated 122 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:31:36.479370 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44df94f1-6d8a-4d7e-830e-3cacbaad93a7 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:36.599411 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44df94f1-6d8a-4d7e-830e-3cacbaad93a7 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:36.638121 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fda17a-ee60-423f-a8a2-59723054c62d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:36.638793 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fda17a-ee60-423f-a8a2-59723054c62d None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:36.639074 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fda17a-ee60-423f-a8a2-59723054c62d None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:36.639348 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fda17a-ee60-423f-a8a2-59723054c62d None admin] PATH_INFO: `/v3/services` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:36.640132 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fda17a-ee60-423f-a8a2-59723054c62d None admin] RBAC: Adding query filter params (type=image) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:36.641006 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fda17a-ee60-423f-a8a2-59723054c62d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:36.642147 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fda17a-ee60-423f-a8a2-59723054c62d None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:36.647277 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 490/994] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jun 16 17:31:36 2022] GET /identity/v3/services?type=image => generated 122 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:37.028595 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bced4592-3924-4c3a-912d-49bd4c60e6f0 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:37.029137 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bced4592-3924-4c3a-912d-49bd4c60e6f0 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:37.029137 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bced4592-3924-4c3a-912d-49bd4c60e6f0 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:37.030747 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 505/995] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:31:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:31:37.035958 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b24823-4e9c-43a7-bef8-24e00b3f65de None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:37.036296 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b24823-4e9c-43a7-bef8-24e00b3f65de None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:37.036642 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b24823-4e9c-43a7-bef8-24e00b3f65de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:37.053803 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-77b24823-4e9c-43a7-bef8-24e00b3f65de None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:31:37.055071 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-77b24823-4e9c-43a7-bef8-24e00b3f65de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:31:37.067330 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:37.131057 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 491/996] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:31:37 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:31:37.154098 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bced4592-3924-4c3a-912d-49bd4c60e6f0 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:37.284009 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bced4592-3924-4c3a-912d-49bd4c60e6f0 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:37.321568 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d5586d-87ea-4ee7-aa5e-0282962c20cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:37.322366 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d5586d-87ea-4ee7-aa5e-0282962c20cf None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:37.322649 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d5586d-87ea-4ee7-aa5e-0282962c20cf None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:37.322925 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d5586d-87ea-4ee7-aa5e-0282962c20cf None admin] PATH_INFO: `/v3/services/compute` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:37.323572 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d5586d-87ea-4ee7-aa5e-0282962c20cf None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:37.324969 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d5586d-87ea-4ee7-aa5e-0282962c20cf None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:37.337010 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-b9d5586d-87ea-4ee7-aa5e-0282962c20cf None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 17:31:37.337010 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:37.337010 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 17:31:37.337010 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 17:31:37.337010 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 17:31:37.337010 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 17:31:37.337010 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 17:31:37.337010 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:31:37.337010 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 17:31:37.337010 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:31:37.337010 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:31:37.337010 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:31:37.337010 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:37.337010 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:31:37.337010 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:31:37.337010 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:31:37.337010 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:31:37.337980 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:31:37.337980 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:31:37.337980 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:31:37.337980 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:31:37.337980 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:31:37.337980 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:31:37.337980 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 17:31:37.337980 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 17:31:37.337980 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 17:31:37.337980 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 17:31:37.337980 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:31:37.337980 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:31:37.337980 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:31:37.337980 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:31:37.337980 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:31:37.337980 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:31:37.338840 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:31:37.338840 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:31:37.338840 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:31:37.338840 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:31:37.338840 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:31:37.338840 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:31:37.338840 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:31:37.338840 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:31:37.338840 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:31:37.338840 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:31:37.338840 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 17:31:37.338840 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:31:37.338840 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 17:31:37.338840 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:31:37.340309 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 506/997] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jun 16 17:31:37 2022] GET /identity/v3/services/compute => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:31:37.345354 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b24823-4e9c-43a7-bef8-24e00b3f65de None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:37.445686 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b24823-4e9c-43a7-bef8-24e00b3f65de None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:37.480064 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a180ea-cfce-4bcf-a868-b5714683c174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:37.480761 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a180ea-cfce-4bcf-a868-b5714683c174 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:37.481047 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a180ea-cfce-4bcf-a868-b5714683c174 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:37.481324 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a180ea-cfce-4bcf-a868-b5714683c174 None admin] PATH_INFO: `/v3/services` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:37.482033 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a180ea-cfce-4bcf-a868-b5714683c174 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:37.482340 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a180ea-cfce-4bcf-a868-b5714683c174 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:37.484260 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a180ea-cfce-4bcf-a868-b5714683c174 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:37.489448 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 492/998] 10.222.0.42 () {56 vars in 1112 bytes} [Thu Jun 16 17:31:37 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:37.494819 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d5586d-87ea-4ee7-aa5e-0282962c20cf None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:37.520966 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:37.618094 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d5586d-87ea-4ee7-aa5e-0282962c20cf None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:37.657451 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20193352-c35c-46af-b805-0ed706d6f22a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:37.658119 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20193352-c35c-46af-b805-0ed706d6f22a None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:37.658396 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20193352-c35c-46af-b805-0ed706d6f22a None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:37.658672 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20193352-c35c-46af-b805-0ed706d6f22a None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:37.659369 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20193352-c35c-46af-b805-0ed706d6f22a None admin] RBAC: Adding query filter params (type=compute) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:37.659672 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20193352-c35c-46af-b805-0ed706d6f22a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:37.661078 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20193352-c35c-46af-b805-0ed706d6f22a None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:37.671563 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 507/999] 10.222.0.42 () {56 vars in 1112 bytes} [Thu Jun 16 17:31:37 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:37.673384 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a180ea-cfce-4bcf-a868-b5714683c174 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:37.686123 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:37.778137 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a180ea-cfce-4bcf-a868-b5714683c174 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:37.813623 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ac7cc8-3206-44c6-bdb9-d5ef8a03cca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:37.814595 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ac7cc8-3206-44c6-bdb9-d5ef8a03cca3 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:37.815052 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ac7cc8-3206-44c6-bdb9-d5ef8a03cca3 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:37.815497 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ac7cc8-3206-44c6-bdb9-d5ef8a03cca3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:37.816487 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ac7cc8-3206-44c6-bdb9-d5ef8a03cca3 None admin] RBAC: Adding query filter params (interface=public, service_id=ad8d0fbef6c94b30be8fc80437dce017, region_id=RegionOne) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:37.817096 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ac7cc8-3206-44c6-bdb9-d5ef8a03cca3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:37.818627 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ac7cc8-3206-44c6-bdb9-d5ef8a03cca3 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:37.824343 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 493/1000] 10.222.0.42 () {56 vars in 1251 bytes} [Thu Jun 16 17:31:37 2022] GET /identity/v3/endpoints?service_id=ad8d0fbef6c94b30be8fc80437dce017&interface=public®ion_id=RegionOne => generated 194 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:37.829782 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20193352-c35c-46af-b805-0ed706d6f22a None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:37.927101 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:37.941014 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20193352-c35c-46af-b805-0ed706d6f22a None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:37.975892 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08998991-4e49-470c-9fbe-b7a98b61d617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:37.976589 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08998991-4e49-470c-9fbe-b7a98b61d617 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:37.976877 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08998991-4e49-470c-9fbe-b7a98b61d617 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:37.977164 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08998991-4e49-470c-9fbe-b7a98b61d617 None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:37.977843 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08998991-4e49-470c-9fbe-b7a98b61d617 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:37.979234 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08998991-4e49-470c-9fbe-b7a98b61d617 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:37.986691 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 508/1001] 10.222.0.42 () {56 vars in 1087 bytes} [Thu Jun 16 17:31:37 2022] GET /identity/v3/services => generated 1727 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 17:31:38.712038 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc50cb10-f0e8-4713-986a-891c7f39abbf None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:38.712599 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc50cb10-f0e8-4713-986a-891c7f39abbf None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:38.712599 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc50cb10-f0e8-4713-986a-891c7f39abbf None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:38.714179 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 494/1002] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:31:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:31:38.720423 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce595e1-25a7-411f-8851-19292181f859 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:38.720935 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce595e1-25a7-411f-8851-19292181f859 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:38.721012 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce595e1-25a7-411f-8851-19292181f859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:38.739431 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-9ce595e1-25a7-411f-8851-19292181f859 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:31:38.740700 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-9ce595e1-25a7-411f-8851-19292181f859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:31:38.795198 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee748d5b-ea68-4e80-b283-c86cfcd28fa0 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:38.795486 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee748d5b-ea68-4e80-b283-c86cfcd28fa0 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:38.795771 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee748d5b-ea68-4e80-b283-c86cfcd28fa0 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:38.797586 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 495/1003] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:31:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:31:38.803390 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c4b16b-2a5f-4a91-bfb8-f0565a23b2f3 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:38.803685 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c4b16b-2a5f-4a91-bfb8-f0565a23b2f3 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:38.804238 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c4b16b-2a5f-4a91-bfb8-f0565a23b2f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:38.814360 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 509/1004] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:31:38 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:31:38.823018 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-14c4b16b-2a5f-4a91-bfb8-f0565a23b2f3 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:31:38.824213 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-14c4b16b-2a5f-4a91-bfb8-f0565a23b2f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:31:38.838039 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce595e1-25a7-411f-8851-19292181f859 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:38.906054 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 496/1005] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:31:38 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:31:38.928362 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c4b16b-2a5f-4a91-bfb8-f0565a23b2f3 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:38.931317 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:38.933908 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:38.955318 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce595e1-25a7-411f-8851-19292181f859 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:38.990081 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21accb20-3b2f-451a-9789-2c81bd083bcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:38.990882 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21accb20-3b2f-451a-9789-2c81bd083bcd None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:38.991165 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21accb20-3b2f-451a-9789-2c81bd083bcd None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:38.991434 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21accb20-3b2f-451a-9789-2c81bd083bcd None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:38.992076 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21accb20-3b2f-451a-9789-2c81bd083bcd None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:38.993467 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21accb20-3b2f-451a-9789-2c81bd083bcd None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:39.005958 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-21accb20-3b2f-451a-9789-2c81bd083bcd None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 17:31:39.005958 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:39.005958 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 17:31:39.005958 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 17:31:39.005958 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 17:31:39.005958 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 17:31:39.005958 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 17:31:39.005958 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:31:39.005958 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 17:31:39.005958 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:31:39.005958 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:31:39.005958 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:31:39.005958 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:39.005958 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:31:39.005958 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:31:39.005958 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:31:39.005958 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:31:39.006986 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:31:39.006986 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:31:39.006986 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:31:39.006986 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:31:39.006986 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:31:39.006986 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:31:39.006986 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 17:31:39.006986 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 17:31:39.006986 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 17:31:39.006986 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 17:31:39.006986 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:31:39.006986 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:31:39.006986 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:31:39.006986 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:31:39.006986 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:31:39.006986 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:31:39.007845 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:31:39.007845 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:31:39.007845 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:31:39.007845 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:31:39.007845 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:31:39.007845 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:31:39.007845 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:31:39.007845 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:31:39.007845 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:31:39.007845 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:31:39.007845 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 17:31:39.007845 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:31:39.007845 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 17:31:39.007845 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:31:39.007845 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 510/1006] 10.222.0.42 () {56 vars in 1126 bytes} [Thu Jun 16 17:31:38 2022] GET /identity/v3/services/object-store => generated 93 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:31:39.013283 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21accb20-3b2f-451a-9789-2c81bd083bcd None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:39.036564 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c4b16b-2a5f-4a91-bfb8-f0565a23b2f3 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:39.072290 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84938fca-cf06-4801-a928-1d601cf9e217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:39.072986 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84938fca-cf06-4801-a928-1d601cf9e217 None admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:39.073264 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84938fca-cf06-4801-a928-1d601cf9e217 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:39.073540 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84938fca-cf06-4801-a928-1d601cf9e217 None admin] PATH_INFO: `/v3/services` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:39.074251 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84938fca-cf06-4801-a928-1d601cf9e217 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:39.075605 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84938fca-cf06-4801-a928-1d601cf9e217 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:39.096851 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 497/1007] 10.222.0.42 () {60 vars in 1141 bytes} [Thu Jun 16 17:31:38 2022] POST /identity/v3/services => generated 242 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:31:39.131345 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21accb20-3b2f-451a-9789-2c81bd083bcd None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:39.178355 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea00d32-5d03-4d69-a7a9-2df4644f0df5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:39.179045 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea00d32-5d03-4d69-a7a9-2df4644f0df5 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:39.179326 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea00d32-5d03-4d69-a7a9-2df4644f0df5 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:39.179599 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea00d32-5d03-4d69-a7a9-2df4644f0df5 None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:39.180310 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea00d32-5d03-4d69-a7a9-2df4644f0df5 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:39.180649 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea00d32-5d03-4d69-a7a9-2df4644f0df5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:39.182049 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea00d32-5d03-4d69-a7a9-2df4644f0df5 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:39.192418 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 511/1008] 10.222.0.42 () {56 vars in 1122 bytes} [Thu Jun 16 17:31:39 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:31:39.195837 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84938fca-cf06-4801-a928-1d601cf9e217 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:39.267764 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:39.311360 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84938fca-cf06-4801-a928-1d601cf9e217 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:39.347197 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a51dc4-6cee-48d3-82f2-880455e50f53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:39.347865 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a51dc4-6cee-48d3-82f2-880455e50f53 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:39.348145 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a51dc4-6cee-48d3-82f2-880455e50f53 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:39.348441 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a51dc4-6cee-48d3-82f2-880455e50f53 None admin] PATH_INFO: `/v3/services` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:39.349145 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a51dc4-6cee-48d3-82f2-880455e50f53 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:39.349445 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a51dc4-6cee-48d3-82f2-880455e50f53 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:39.350810 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a51dc4-6cee-48d3-82f2-880455e50f53 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:39.356076 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 498/1009] 10.222.0.42 () {56 vars in 1122 bytes} [Thu Jun 16 17:31:39 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:39.362176 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea00d32-5d03-4d69-a7a9-2df4644f0df5 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:39.371707 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:39.468239 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea00d32-5d03-4d69-a7a9-2df4644f0df5 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:39.502670 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e40135-1930-4745-a0dc-bf1125c1249d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:39.503333 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e40135-1930-4745-a0dc-bf1125c1249d None admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:39.503614 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e40135-1930-4745-a0dc-bf1125c1249d None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:39.503886 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e40135-1930-4745-a0dc-bf1125c1249d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:39.504649 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e40135-1930-4745-a0dc-bf1125c1249d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:39.506011 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e40135-1930-4745-a0dc-bf1125c1249d None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:39.528484 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 512/1010] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Jun 16 17:31:39 2022] POST /identity/v3/endpoints => generated 325 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:31:40.198894 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74ab354-7f68-4b82-85ad-72bf40b405c1 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:40.199422 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74ab354-7f68-4b82-85ad-72bf40b405c1 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:40.199422 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74ab354-7f68-4b82-85ad-72bf40b405c1 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:40.201002 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 499/1011] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:31:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:31:40.206655 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909951d4-f8fb-4dbe-af81-d4a9b4ab7256 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:40.207072 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909951d4-f8fb-4dbe-af81-d4a9b4ab7256 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:40.207240 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909951d4-f8fb-4dbe-af81-d4a9b4ab7256 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:40.225326 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-909951d4-f8fb-4dbe-af81-d4a9b4ab7256 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:31:40.226553 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-909951d4-f8fb-4dbe-af81-d4a9b4ab7256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:31:40.289493 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:40.305875 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 513/1012] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:31:40 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:31:40.328543 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74ab354-7f68-4b82-85ad-72bf40b405c1 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:40.431715 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74ab354-7f68-4b82-85ad-72bf40b405c1 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:40.457364 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:40.465947 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb2e6d3-2715-476e-9e62-d14038d1f5d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:40.466735 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb2e6d3-2715-476e-9e62-d14038d1f5d2 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:40.467015 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb2e6d3-2715-476e-9e62-d14038d1f5d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:40.467287 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb2e6d3-2715-476e-9e62-d14038d1f5d2 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:40.468465 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb2e6d3-2715-476e-9e62-d14038d1f5d2 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:40.470281 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb2e6d3-2715-476e-9e62-d14038d1f5d2 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:40.482362 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-5cb2e6d3-2715-476e-9e62-d14038d1f5d2 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 17:31:40.482362 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:40.482362 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 17:31:40.482362 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 17:31:40.482362 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 17:31:40.482362 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 17:31:40.482362 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 17:31:40.482362 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:31:40.482362 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 17:31:40.482362 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:31:40.482362 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:31:40.482362 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:31:40.482362 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:40.482362 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:31:40.482362 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:31:40.482362 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:31:40.482362 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:31:40.483357 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:31:40.483357 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:31:40.483357 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:31:40.483357 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:31:40.483357 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:31:40.483357 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:31:40.483357 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 17:31:40.483357 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 17:31:40.483357 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 17:31:40.483357 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 17:31:40.483357 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:31:40.483357 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:31:40.483357 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:31:40.483357 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:31:40.483357 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:31:40.483357 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:31:40.485087 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:31:40.485087 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:31:40.485087 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:31:40.485087 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:31:40.485087 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:31:40.485087 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:31:40.485087 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:31:40.485087 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:31:40.485087 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:31:40.485087 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:31:40.485087 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 17:31:40.485087 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:31:40.485087 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 17:31:40.485087 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:31:40.488745 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 500/1013] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jun 16 17:31:40 2022] GET /identity/v3/services/compute => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:31:40.490689 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909951d4-f8fb-4dbe-af81-d4a9b4ab7256 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:40.593527 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909951d4-f8fb-4dbe-af81-d4a9b4ab7256 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:40.627566 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2310e23-e6c8-4200-8ccb-9092a714ac45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:40.628256 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2310e23-e6c8-4200-8ccb-9092a714ac45 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:40.628570 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2310e23-e6c8-4200-8ccb-9092a714ac45 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:40.628850 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2310e23-e6c8-4200-8ccb-9092a714ac45 None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:40.629572 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2310e23-e6c8-4200-8ccb-9092a714ac45 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:40.629882 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2310e23-e6c8-4200-8ccb-9092a714ac45 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:40.631297 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2310e23-e6c8-4200-8ccb-9092a714ac45 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:40.642467 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 514/1014] 10.222.0.42 () {56 vars in 1112 bytes} [Thu Jun 16 17:31:40 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:40.643566 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb2e6d3-2715-476e-9e62-d14038d1f5d2 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:40.749989 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb2e6d3-2715-476e-9e62-d14038d1f5d2 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:40.784740 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffc726a-d3a8-4dc7-a426-83a2182ed5bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:40.785412 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffc726a-d3a8-4dc7-a426-83a2182ed5bd None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:40.785699 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffc726a-d3a8-4dc7-a426-83a2182ed5bd None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:40.785974 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffc726a-d3a8-4dc7-a426-83a2182ed5bd None admin] PATH_INFO: `/v3/services` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:40.786679 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffc726a-d3a8-4dc7-a426-83a2182ed5bd None admin] RBAC: Adding query filter params (type=compute) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:40.786986 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffc726a-d3a8-4dc7-a426-83a2182ed5bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:40.788350 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffc726a-d3a8-4dc7-a426-83a2182ed5bd None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:40.798028 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 501/1015] 10.222.0.42 () {56 vars in 1112 bytes} [Thu Jun 16 17:31:40 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:40.799140 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2310e23-e6c8-4200-8ccb-9092a714ac45 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:40.872240 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:40.927215 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2310e23-e6c8-4200-8ccb-9092a714ac45 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:40.961458 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3387c963-2eb5-4711-9fcc-bd24f0bb4b8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:40.962119 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3387c963-2eb5-4711-9fcc-bd24f0bb4b8e None admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:40.962403 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3387c963-2eb5-4711-9fcc-bd24f0bb4b8e None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:40.962682 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3387c963-2eb5-4711-9fcc-bd24f0bb4b8e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:40.963422 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3387c963-2eb5-4711-9fcc-bd24f0bb4b8e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:40.965617 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3387c963-2eb5-4711-9fcc-bd24f0bb4b8e None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:40.992416 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 515/1016] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Jun 16 17:31:40 2022] POST /identity/v3/endpoints => generated 334 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:31:41.362644 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6f0e88-8b51-4e56-a57b-4b5b5a8c57fd None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:41.363168 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6f0e88-8b51-4e56-a57b-4b5b5a8c57fd None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:41.363245 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6f0e88-8b51-4e56-a57b-4b5b5a8c57fd None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:41.364623 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 502/1017] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:31:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:31:41.369921 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8cefed-a286-4c18-a6b6-0f6f68ba3af0 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:41.370343 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8cefed-a286-4c18-a6b6-0f6f68ba3af0 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:41.370511 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8cefed-a286-4c18-a6b6-0f6f68ba3af0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:41.388270 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-6d8cefed-a286-4c18-a6b6-0f6f68ba3af0 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:31:41.389487 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-6d8cefed-a286-4c18-a6b6-0f6f68ba3af0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:31:41.465653 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 516/1018] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:31:41 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:31:41.489819 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6f0e88-8b51-4e56-a57b-4b5b5a8c57fd None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:41.550610 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:41.597808 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6f0e88-8b51-4e56-a57b-4b5b5a8c57fd None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:41.632435 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd89e4e-b7b6-4865-aa4c-f2438c0ca069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:41.633154 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd89e4e-b7b6-4865-aa4c-f2438c0ca069 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:41.633429 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd89e4e-b7b6-4865-aa4c-f2438c0ca069 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:41.633699 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd89e4e-b7b6-4865-aa4c-f2438c0ca069 None admin] PATH_INFO: `/v3/services/image` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:41.634344 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd89e4e-b7b6-4865-aa4c-f2438c0ca069 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:41.635709 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd89e4e-b7b6-4865-aa4c-f2438c0ca069 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:41.647868 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-1dd89e4e-b7b6-4865-aa4c-f2438c0ca069 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 17:31:41.647868 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:41.647868 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 17:31:41.647868 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 17:31:41.647868 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 17:31:41.647868 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 17:31:41.647868 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 17:31:41.647868 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:31:41.647868 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 17:31:41.647868 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:31:41.647868 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:31:41.647868 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:31:41.647868 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:41.647868 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:31:41.647868 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:31:41.647868 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:31:41.647868 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:31:41.649211 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:31:41.649211 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:31:41.649211 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:31:41.649211 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:31:41.649211 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:31:41.649211 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:31:41.649211 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 17:31:41.649211 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 17:31:41.649211 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 17:31:41.649211 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 17:31:41.649211 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:31:41.649211 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:31:41.649211 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:31:41.649211 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:31:41.649211 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:31:41.649211 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:31:41.650133 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:31:41.650133 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:31:41.650133 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:31:41.650133 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:31:41.650133 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:31:41.650133 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:31:41.650133 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:31:41.650133 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:31:41.650133 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:31:41.650133 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:31:41.650133 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 17:31:41.650133 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:31:41.650133 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 17:31:41.650133 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:31:41.650133 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 503/1019] 10.222.0.42 () {56 vars in 1105 bytes} [Thu Jun 16 17:31:41 2022] GET /identity/v3/services/image => generated 86 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:31:41.658327 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8cefed-a286-4c18-a6b6-0f6f68ba3af0 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:41.700164 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:41.721162 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228fb0ef-5dde-4845-a361-0bd055c02118 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:41.721481 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228fb0ef-5dde-4845-a361-0bd055c02118 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:41.721726 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228fb0ef-5dde-4845-a361-0bd055c02118 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:41.723096 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 504/1020] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:31:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:31:41.728104 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d232e9-9368-4249-ab92-181f18a48016 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:41.728421 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d232e9-9368-4249-ab92-181f18a48016 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:41.728766 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d232e9-9368-4249-ab92-181f18a48016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:41.746007 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-a3d232e9-9368-4249-ab92-181f18a48016 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:31:41.747188 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-a3d232e9-9368-4249-ab92-181f18a48016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:31:41.775930 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8cefed-a286-4c18-a6b6-0f6f68ba3af0 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:41.812752 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670e2d4f-44c7-4468-b026-1936df4603be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:41.813425 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670e2d4f-44c7-4468-b026-1936df4603be None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:41.813706 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670e2d4f-44c7-4468-b026-1936df4603be None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:41.813980 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670e2d4f-44c7-4468-b026-1936df4603be None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:41.814686 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670e2d4f-44c7-4468-b026-1936df4603be None admin] RBAC: Adding query filter params (name=image) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:41.814992 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670e2d4f-44c7-4468-b026-1936df4603be None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:41.816362 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670e2d4f-44c7-4468-b026-1936df4603be None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:41.820563 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 505/1021] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:31:41 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 17:31:41.821559 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 517/1022] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jun 16 17:31:41 2022] GET /identity/v3/services?name=image => generated 122 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:41.826895 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d232e9-9368-4249-ab92-181f18a48016 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:41.846366 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670e2d4f-44c7-4468-b026-1936df4603be None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:41.886732 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:41.944282 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d232e9-9368-4249-ab92-181f18a48016 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:41.971493 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670e2d4f-44c7-4468-b026-1936df4603be None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:41.973194 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:41.982752 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37264d2-2984-4021-9eac-6a2ad7bd11f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:41.983416 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37264d2-2984-4021-9eac-6a2ad7bd11f9 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:41.984035 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37264d2-2984-4021-9eac-6a2ad7bd11f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:41.984589 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37264d2-2984-4021-9eac-6a2ad7bd11f9 None admin] PATH_INFO: `/v3/services` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:41.985308 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37264d2-2984-4021-9eac-6a2ad7bd11f9 None admin] RBAC: Adding query filter params (type=image) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:41.985618 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37264d2-2984-4021-9eac-6a2ad7bd11f9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:41.986982 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37264d2-2984-4021-9eac-6a2ad7bd11f9 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:41.992103 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 506/1023] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jun 16 17:31:41 2022] GET /identity/v3/services?type=image => generated 350 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:41.997617 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37264d2-2984-4021-9eac-6a2ad7bd11f9 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:42.010986 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc0a8f2-de21-4a67-b9e6-53ebb2ce0e64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:42.011840 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc0a8f2-de21-4a67-b9e6-53ebb2ce0e64 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:42.012117 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc0a8f2-de21-4a67-b9e6-53ebb2ce0e64 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:42.012477 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc0a8f2-de21-4a67-b9e6-53ebb2ce0e64 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:42.013705 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc0a8f2-de21-4a67-b9e6-53ebb2ce0e64 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:42.016174 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc0a8f2-de21-4a67-b9e6-53ebb2ce0e64 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:42.018041 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 518/1024] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jun 16 17:31:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:31:42.023639 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc0a8f2-de21-4a67-b9e6-53ebb2ce0e64 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:42.101234 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37264d2-2984-4021-9eac-6a2ad7bd11f9 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:42.115437 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:42.129499 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc0a8f2-de21-4a67-b9e6-53ebb2ce0e64 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:42.137031 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2fdf16-5bff-4e11-8e49-83cb1915e2f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:42.138034 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2fdf16-5bff-4e11-8e49-83cb1915e2f4 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:42.138317 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2fdf16-5bff-4e11-8e49-83cb1915e2f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:42.138588 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2fdf16-5bff-4e11-8e49-83cb1915e2f4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:42.139726 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2fdf16-5bff-4e11-8e49-83cb1915e2f4 None admin] RBAC: Adding query filter params (interface=public, service_id=6b3c8c96a1004c439a9ad9c5f3378b4b, region_id=RegionOne) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:42.140154 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2fdf16-5bff-4e11-8e49-83cb1915e2f4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:42.142677 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2fdf16-5bff-4e11-8e49-83cb1915e2f4 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:42.148100 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 507/1025] 10.222.0.42 () {56 vars in 1251 bytes} [Thu Jun 16 17:31:41 2022] GET /identity/v3/endpoints?service_id=6b3c8c96a1004c439a9ad9c5f3378b4b&interface=public®ion_id=RegionOne => generated 194 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:31:42.153464 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2fdf16-5bff-4e11-8e49-83cb1915e2f4 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:42.167663 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f90a02-0e0f-4cf0-8660-11701d7c7be2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:42.168394 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f90a02-0e0f-4cf0-8660-11701d7c7be2 None admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:42.168713 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f90a02-0e0f-4cf0-8660-11701d7c7be2 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:42.168994 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f90a02-0e0f-4cf0-8660-11701d7c7be2 None admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:42.169712 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f90a02-0e0f-4cf0-8660-11701d7c7be2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:42.171809 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f90a02-0e0f-4cf0-8660-11701d7c7be2 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:42.206162 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 519/1026] 10.222.0.42 () {60 vars in 1141 bytes} [Thu Jun 16 17:31:42 2022] POST /identity/v3/projects => generated 311 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:31:42.267411 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2fdf16-5bff-4e11-8e49-83cb1915e2f4 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:42.311161 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c93e5d-c698-4f5b-8a4f-c731970536a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:42.312432 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c93e5d-c698-4f5b-8a4f-c731970536a4 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:42.312726 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c93e5d-c698-4f5b-8a4f-c731970536a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:42.313007 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c93e5d-c698-4f5b-8a4f-c731970536a4 None admin] PATH_INFO: `/v3/services` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:42.313809 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c93e5d-c698-4f5b-8a4f-c731970536a4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:42.315203 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c93e5d-c698-4f5b-8a4f-c731970536a4 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:42.323896 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 508/1027] 10.222.0.42 () {56 vars in 1087 bytes} [Thu Jun 16 17:31:42 2022] GET /identity/v3/services => generated 1957 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 17:31:43.269730 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa8ddc6-8ae8-4438-bb89-248dee34ccf1 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:43.270360 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa8ddc6-8ae8-4438-bb89-248dee34ccf1 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:43.270360 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa8ddc6-8ae8-4438-bb89-248dee34ccf1 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:43.271797 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 520/1028] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:31:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:31:43.276652 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96686b53-9fed-4bdf-b06b-92ac10b88597 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:43.277069 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96686b53-9fed-4bdf-b06b-92ac10b88597 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:43.277252 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96686b53-9fed-4bdf-b06b-92ac10b88597 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:43.294771 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-96686b53-9fed-4bdf-b06b-92ac10b88597 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:31:43.295941 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-96686b53-9fed-4bdf-b06b-92ac10b88597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:31:43.333025 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:43.371038 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 509/1029] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:31:43 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:31:43.390869 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa8ddc6-8ae8-4438-bb89-248dee34ccf1 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:43.416945 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:43.502010 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa8ddc6-8ae8-4438-bb89-248dee34ccf1 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:43.538152 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14414525-fd1c-4f92-ad8e-0f7d52331edd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:43.539211 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14414525-fd1c-4f92-ad8e-0f7d52331edd None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:43.539658 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14414525-fd1c-4f92-ad8e-0f7d52331edd None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:43.540086 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14414525-fd1c-4f92-ad8e-0f7d52331edd None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:43.551563 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14414525-fd1c-4f92-ad8e-0f7d52331edd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:31:43.559958 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-14414525-fd1c-4f92-ad8e-0f7d52331edd None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 17:31:43.559958 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:43.559958 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:31:43.559958 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:31:43.559958 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:31:43.559958 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:31:43.559958 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:31:43.559958 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:31:43.559958 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:31:43.559958 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:31:43.559958 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:31:43.559958 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:31:43.559958 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:31:43.559958 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:31:43.559958 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:31:43.559958 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:31:43.561115 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:31:43.561115 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:31:43.561115 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:31:43.561115 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:31:43.561115 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:31:43.561115 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:31:43.561115 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:31:43.561115 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:31:43.561115 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:31:43.561115 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:31:43.561115 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:31:43.561115 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:31:43.561115 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:31:43.561115 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:31:43.561115 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:31:43.561115 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:31:43.562061 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:31:43.562061 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:31:43.562061 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:31:43.562061 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:31:43.562061 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:31:43.562061 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:31:43.562061 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:31:43.562061 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:31:43.562061 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:31:43.562061 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:31:43.562061 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 17:31:43.562061 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:31:43.567803 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 521/1030] 10.222.0.42 () {56 vars in 1120 bytes} [Thu Jun 16 17:31:43 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:31:43.569726 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96686b53-9fed-4bdf-b06b-92ac10b88597 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:43.675793 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96686b53-9fed-4bdf-b06b-92ac10b88597 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:43.712832 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10309350-2f80-4dd9-8ca7-5154f05a7c63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:43.713834 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10309350-2f80-4dd9-8ca7-5154f05a7c63 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:43.714320 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10309350-2f80-4dd9-8ca7-5154f05a7c63 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:43.714783 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10309350-2f80-4dd9-8ca7-5154f05a7c63 None admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:43.715711 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10309350-2f80-4dd9-8ca7-5154f05a7c63 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:43.716223 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10309350-2f80-4dd9-8ca7-5154f05a7c63 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:43.717884 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10309350-2f80-4dd9-8ca7-5154f05a7c63 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:43.731618 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 510/1031] 10.222.0.42 () {56 vars in 1115 bytes} [Thu Jun 16 17:31:43 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:43.733159 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14414525-fd1c-4f92-ad8e-0f7d52331edd None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:43.821744 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:43.841759 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14414525-fd1c-4f92-ad8e-0f7d52331edd None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:43.878112 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd14d832-d3cf-4fa8-9761-cc1c794befbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:43.878909 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd14d832-d3cf-4fa8-9761-cc1c794befbf None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:43.879196 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd14d832-d3cf-4fa8-9761-cc1c794befbf None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:43.879477 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd14d832-d3cf-4fa8-9761-cc1c794befbf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:43.895120 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd14d832-d3cf-4fa8-9761-cc1c794befbf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:43.896612 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd14d832-d3cf-4fa8-9761-cc1c794befbf None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:43.914677 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-fd14d832-d3cf-4fa8-9761-cc1c794befbf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:31:43.914677 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:43.914677 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:31:43.914677 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:31:43.914677 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:31:43.914677 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:31:43.914677 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:31:43.914677 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:31:43.914677 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:31:43.914677 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:31:43.914677 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:31:43.914677 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:31:43.914677 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:31:43.914677 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:31:43.914677 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:31:43.915657 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:31:43.915657 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:31:43.915657 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:31:43.915657 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:31:43.915657 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:31:43.915657 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:31:43.915657 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:31:43.915657 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:31:43.915657 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:31:43.915657 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:31:43.915657 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:31:43.915657 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:31:43.915657 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:31:43.915657 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:31:43.915657 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:31:43.915657 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:31:43.915657 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:31:43.921841 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:31:43.921841 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:31:43.921841 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:31:43.921841 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:31:43.921841 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 522/1032] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jun 16 17:31:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:31:43.923928 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10309350-2f80-4dd9-8ca7-5154f05a7c63 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:43.960926 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:44.034652 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10309350-2f80-4dd9-8ca7-5154f05a7c63 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:44.070067 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf92730-2b7d-4265-bb1b-4eac5618d189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:44.070741 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf92730-2b7d-4265-bb1b-4eac5618d189 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:44.071025 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf92730-2b7d-4265-bb1b-4eac5618d189 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:44.071311 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf92730-2b7d-4265-bb1b-4eac5618d189 None admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:44.072132 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf92730-2b7d-4265-bb1b-4eac5618d189 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:44.072321 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf92730-2b7d-4265-bb1b-4eac5618d189 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:44.073720 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf92730-2b7d-4265-bb1b-4eac5618d189 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:44.087029 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 511/1033] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jun 16 17:31:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:44.092726 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd14d832-d3cf-4fa8-9761-cc1c794befbf None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:44.198472 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd14d832-d3cf-4fa8-9761-cc1c794befbf None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:44.231982 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe8f35a-2f01-492a-874e-cd21a2f7b4cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:44.232802 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe8f35a-2f01-492a-874e-cd21a2f7b4cf None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:44.233090 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe8f35a-2f01-492a-874e-cd21a2f7b4cf None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:44.233369 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe8f35a-2f01-492a-874e-cd21a2f7b4cf None admin] PATH_INFO: `/v3/users/nova` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:44.235854 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:44.243417 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe8f35a-2f01-492a-874e-cd21a2f7b4cf None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:44.244899 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe8f35a-2f01-492a-874e-cd21a2f7b4cf None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:44.259098 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-cbe8f35a-2f01-492a-874e-cd21a2f7b4cf None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 17:31:44.259098 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:44.259098 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:31:44.259098 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:31:44.259098 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:31:44.259098 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:31:44.259098 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:31:44.259098 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:31:44.259098 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:31:44.259098 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:31:44.259098 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:31:44.259098 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:31:44.259098 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:31:44.259098 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:31:44.259098 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:31:44.259098 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:31:44.260089 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:31:44.260089 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:31:44.260089 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:31:44.260089 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:31:44.260089 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:31:44.260089 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:31:44.260089 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:31:44.260089 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:31:44.260089 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:31:44.260089 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:31:44.260089 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:31:44.260089 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:31:44.260089 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:31:44.260089 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:31:44.260089 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:31:44.260089 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:31:44.260089 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:31:44.260089 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:31:44.266186 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:31:44.266186 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:31:44.266186 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:31:44.266186 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:31:44.266186 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:31:44.266186 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:31:44.266186 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:31:44.266186 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:31:44.266186 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 17:31:44.266186 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:31:44.266186 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 523/1034] 10.222.0.42 () {56 vars in 1093 bytes} [Thu Jun 16 17:31:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:31:44.268748 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf92730-2b7d-4265-bb1b-4eac5618d189 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:44.369275 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:44.380573 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf92730-2b7d-4265-bb1b-4eac5618d189 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:44.420069 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5260b2ed-79c7-4198-bcdd-dfae2fb9e1bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:44.420787 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5260b2ed-79c7-4198-bcdd-dfae2fb9e1bf None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:44.421646 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5260b2ed-79c7-4198-bcdd-dfae2fb9e1bf None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:44.422374 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5260b2ed-79c7-4198-bcdd-dfae2fb9e1bf None admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:44.422744 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5260b2ed-79c7-4198-bcdd-dfae2fb9e1bf None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:44.423052 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5260b2ed-79c7-4198-bcdd-dfae2fb9e1bf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:44.424499 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5260b2ed-79c7-4198-bcdd-dfae2fb9e1bf None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:44.446623 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7885392-2366-4761-9fa8-1bbd30b92a70 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:44.447180 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7885392-2366-4761-9fa8-1bbd30b92a70 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:44.447180 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7885392-2366-4761-9fa8-1bbd30b92a70 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:44.447990 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 512/1035] 10.222.0.42 () {56 vars in 1133 bytes} [Thu Jun 16 17:31:44 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:44.448922 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 524/1036] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:31:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:31:44.449499 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5260b2ed-79c7-4198-bcdd-dfae2fb9e1bf None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:44.454671 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f4bf63-77ae-480f-a270-3e4884836374 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:44.454976 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f4bf63-77ae-480f-a270-3e4884836374 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:44.455225 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f4bf63-77ae-480f-a270-3e4884836374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:44.472994 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-b1f4bf63-77ae-480f-a270-3e4884836374 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:31:44.474213 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-b1f4bf63-77ae-480f-a270-3e4884836374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:31:44.544577 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 525/1037] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:31:44 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:31:44.546598 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be3e626-13df-4378-8386-25ab0110ce48 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:44.546722 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be3e626-13df-4378-8386-25ab0110ce48 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:44.547129 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be3e626-13df-4378-8386-25ab0110ce48 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:44.548569 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 526/1038] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:31:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:31:44.553557 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a5dbf6-29b9-4937-9d34-663f2fc0a1e5 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:44.553978 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a5dbf6-29b9-4937-9d34-663f2fc0a1e5 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:44.554153 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a5dbf6-29b9-4937-9d34-663f2fc0a1e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:44.569763 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5260b2ed-79c7-4198-bcdd-dfae2fb9e1bf None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:44.570908 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-09a5dbf6-29b9-4937-9d34-663f2fc0a1e5 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:31:44.572091 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-09a5dbf6-29b9-4937-9d34-663f2fc0a1e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:31:44.606058 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda67a10-21b9-4bb2-b6e9-a809d614a4aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:44.606709 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda67a10-21b9-4bb2-b6e9-a809d614a4aa None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:44.606993 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda67a10-21b9-4bb2-b6e9-a809d614a4aa None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:44.607268 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda67a10-21b9-4bb2-b6e9-a809d614a4aa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:44.629123 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:44.641202 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 527/1039] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:31:44 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 17:31:44.642263 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a5dbf6-29b9-4937-9d34-663f2fc0a1e5 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:44.680541 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda67a10-21b9-4bb2-b6e9-a809d614a4aa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:44.681996 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda67a10-21b9-4bb2-b6e9-a809d614a4aa None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:44.698803 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:44.749639 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a5dbf6-29b9-4937-9d34-663f2fc0a1e5 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:44.784300 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 513/1040] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jun 16 17:31:44 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 16 17:31:44.784816 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d987e674-b4a8-4417-beb4-038e1989e920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:44.785181 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda67a10-21b9-4bb2-b6e9-a809d614a4aa None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:44.785607 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d987e674-b4a8-4417-beb4-038e1989e920 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:44.785892 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d987e674-b4a8-4417-beb4-038e1989e920 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:44.786165 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d987e674-b4a8-4417-beb4-038e1989e920 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:44.787509 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d987e674-b4a8-4417-beb4-038e1989e920 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:44.789760 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d987e674-b4a8-4417-beb4-038e1989e920 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:44.795785 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 528/1041] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jun 16 17:31:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:44.795785 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d987e674-b4a8-4417-beb4-038e1989e920 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:44.836584 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:44.886311 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda67a10-21b9-4bb2-b6e9-a809d614a4aa None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:44.901060 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d987e674-b4a8-4417-beb4-038e1989e920 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:44.922170 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b12f9cf-0c22-42cb-8a09-926fa0874910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:44.922956 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b12f9cf-0c22-42cb-8a09-926fa0874910 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:44.923238 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b12f9cf-0c22-42cb-8a09-926fa0874910 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:44.923511 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b12f9cf-0c22-42cb-8a09-926fa0874910 None admin] PATH_INFO: `/v3/services/image` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:44.924156 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b12f9cf-0c22-42cb-8a09-926fa0874910 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:44.924696 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:44.925617 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b12f9cf-0c22-42cb-8a09-926fa0874910 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:44.936258 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6518f2ed-6bc8-4539-a125-16516230ce2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:44.937265 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6518f2ed-6bc8-4539-a125-16516230ce2b None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:44.937465 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-3b12f9cf-0c22-42cb-8a09-926fa0874910 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 17:31:44.937465 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:44.937465 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 17:31:44.937465 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 17:31:44.937465 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 17:31:44.937465 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 17:31:44.937465 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 17:31:44.937465 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:31:44.937465 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 17:31:44.937465 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:31:44.937465 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:31:44.937465 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:31:44.937465 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:44.937465 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:31:44.937465 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:31:44.937465 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:31:44.937465 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:31:44.938599 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:31:44.938599 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:31:44.938599 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:31:44.938599 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:31:44.938599 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:31:44.938599 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:31:44.938599 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 17:31:44.938599 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 17:31:44.938599 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 17:31:44.938599 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 17:31:44.938599 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:31:44.938599 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:31:44.938599 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:31:44.938599 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:31:44.938599 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:31:44.938599 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:31:44.939468 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:31:44.939468 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:31:44.939468 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:31:44.939468 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:31:44.939468 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:31:44.939468 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:31:44.939468 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:31:44.939468 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:31:44.939468 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:31:44.939468 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:31:44.939468 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 17:31:44.939468 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:31:44.939468 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 17:31:44.939468 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:31:44.940214 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6518f2ed-6bc8-4539-a125-16516230ce2b None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:44.940214 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6518f2ed-6bc8-4539-a125-16516230ce2b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:44.940357 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 514/1042] 10.222.0.42 () {56 vars in 1105 bytes} [Thu Jun 16 17:31:44 2022] GET /identity/v3/services/image => generated 86 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:31:44.940357 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b12f9cf-0c22-42cb-8a09-926fa0874910 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:44.949672 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6518f2ed-6bc8-4539-a125-16516230ce2b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:44.951121 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6518f2ed-6bc8-4539-a125-16516230ce2b None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:44.969159 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-6518f2ed-6bc8-4539-a125-16516230ce2b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:31:44.969159 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:44.969159 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:31:44.969159 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:31:44.969159 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:31:44.969159 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:31:44.969159 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:31:44.969159 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:31:44.969159 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:31:44.969159 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:31:44.969159 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:31:44.969159 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:31:44.969159 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:31:44.969159 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:31:44.969159 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:31:44.970378 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:31:44.970378 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:31:44.970378 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:31:44.970378 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:31:44.970378 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:31:44.970378 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:31:44.970378 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:31:44.970378 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:31:44.970378 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:31:44.970378 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:31:44.970378 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:31:44.970378 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:31:44.970378 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:31:44.970378 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:31:44.970378 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:31:44.970378 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:31:44.970378 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:31:44.971529 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:31:44.971529 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:31:44.971529 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:31:44.971529 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:31:44.971529 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 529/1043] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jun 16 17:31:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:31:44.971529 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6518f2ed-6bc8-4539-a125-16516230ce2b None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:45.049487 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b12f9cf-0c22-42cb-8a09-926fa0874910 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:45.067760 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:45.094504 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598eccca-f069-4d85-86e4-258cec3be6ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:45.094504 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598eccca-f069-4d85-86e4-258cec3be6ef None admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:45.094930 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598eccca-f069-4d85-86e4-258cec3be6ef None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:45.095051 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598eccca-f069-4d85-86e4-258cec3be6ef None admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:45.095673 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598eccca-f069-4d85-86e4-258cec3be6ef None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:45.097784 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598eccca-f069-4d85-86e4-258cec3be6ef None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:45.100657 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6518f2ed-6bc8-4539-a125-16516230ce2b None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:45.124540 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 515/1044] 10.222.0.42 () {60 vars in 1132 bytes} [Thu Jun 16 17:31:44 2022] POST /identity/v3/users => generated 285 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:31:45.126088 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598eccca-f069-4d85-86e4-258cec3be6ef None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:45.142762 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfdce0e-13f5-4778-a237-84b1b5729808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:45.143686 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfdce0e-13f5-4778-a237-84b1b5729808 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:45.144144 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfdce0e-13f5-4778-a237-84b1b5729808 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:45.144727 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfdce0e-13f5-4778-a237-84b1b5729808 None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:45.145728 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfdce0e-13f5-4778-a237-84b1b5729808 None admin] RBAC: Adding query filter params (name=image) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:45.146330 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfdce0e-13f5-4778-a237-84b1b5729808 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:45.148182 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfdce0e-13f5-4778-a237-84b1b5729808 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:45.154765 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 530/1045] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jun 16 17:31:44 2022] GET /identity/v3/services?name=image => generated 122 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:45.161349 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfdce0e-13f5-4778-a237-84b1b5729808 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:45.211049 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:45.268943 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598eccca-f069-4d85-86e4-258cec3be6ef None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:45.300996 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfdce0e-13f5-4778-a237-84b1b5729808 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:45.319111 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa20e370-8dd0-4f69-81b5-7e27914b3b91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:45.319111 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa20e370-8dd0-4f69-81b5-7e27914b3b91 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:45.319111 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa20e370-8dd0-4f69-81b5-7e27914b3b91 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:45.319111 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa20e370-8dd0-4f69-81b5-7e27914b3b91 None admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:45.319576 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa20e370-8dd0-4f69-81b5-7e27914b3b91 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:45.319576 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa20e370-8dd0-4f69-81b5-7e27914b3b91 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:45.322120 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa20e370-8dd0-4f69-81b5-7e27914b3b91 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:45.334610 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 516/1046] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jun 16 17:31:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:45.340058 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa20e370-8dd0-4f69-81b5-7e27914b3b91 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:45.351056 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42a174f-94e5-4f5c-9ae3-71b32dc8a51d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:45.351735 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42a174f-94e5-4f5c-9ae3-71b32dc8a51d None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:45.352022 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42a174f-94e5-4f5c-9ae3-71b32dc8a51d None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:45.352298 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42a174f-94e5-4f5c-9ae3-71b32dc8a51d None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:45.353037 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42a174f-94e5-4f5c-9ae3-71b32dc8a51d None admin] RBAC: Adding query filter params (type=image) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:45.353347 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42a174f-94e5-4f5c-9ae3-71b32dc8a51d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:45.354725 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42a174f-94e5-4f5c-9ae3-71b32dc8a51d None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:45.360702 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 531/1047] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jun 16 17:31:45 2022] GET /identity/v3/services?type=image => generated 350 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:45.367292 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42a174f-94e5-4f5c-9ae3-71b32dc8a51d None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:45.382890 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:45.460939 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa20e370-8dd0-4f69-81b5-7e27914b3b91 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:45.492687 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42a174f-94e5-4f5c-9ae3-71b32dc8a51d None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:45.495091 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:45.496615 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35af3f84-8e7b-44d0-a238-32e57aca5379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:45.497716 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35af3f84-8e7b-44d0-a238-32e57aca5379 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:45.497716 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35af3f84-8e7b-44d0-a238-32e57aca5379 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:45.497991 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35af3f84-8e7b-44d0-a238-32e57aca5379 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:45.506996 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35af3f84-8e7b-44d0-a238-32e57aca5379 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:45.508494 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35af3f84-8e7b-44d0-a238-32e57aca5379 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:45.523899 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-35af3f84-8e7b-44d0-a238-32e57aca5379 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:31:45.523899 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:45.523899 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:31:45.523899 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:31:45.523899 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:31:45.523899 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:31:45.523899 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:31:45.523899 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:31:45.523899 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:31:45.523899 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:31:45.523899 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:31:45.523899 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:31:45.523899 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:31:45.523899 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:31:45.523899 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:31:45.524981 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:31:45.524981 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:31:45.524981 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:31:45.524981 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:31:45.524981 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:31:45.524981 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:31:45.524981 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:31:45.524981 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:31:45.524981 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:31:45.524981 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:31:45.524981 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:31:45.524981 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:31:45.524981 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:31:45.524981 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:31:45.524981 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:31:45.524981 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:31:45.524981 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:31:45.525907 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:31:45.525907 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:31:45.525907 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:31:45.525907 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:31:45.525907 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:31:45.525907 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:31:45.525907 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:31:45.525907 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:31:45.525907 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 517/1048] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jun 16 17:31:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:31:45.531202 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35af3f84-8e7b-44d0-a238-32e57aca5379 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:45.537749 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa162b81-f066-48a0-8c8a-28b6f0f5fb13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:45.538420 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa162b81-f066-48a0-8c8a-28b6f0f5fb13 None admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:45.538709 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa162b81-f066-48a0-8c8a-28b6f0f5fb13 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:45.538987 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa162b81-f066-48a0-8c8a-28b6f0f5fb13 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:45.539742 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa162b81-f066-48a0-8c8a-28b6f0f5fb13 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:45.541193 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa162b81-f066-48a0-8c8a-28b6f0f5fb13 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:45.572408 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 532/1049] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Jun 16 17:31:45 2022] POST /identity/v3/endpoints => generated 327 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:31:45.651357 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:45.662517 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35af3f84-8e7b-44d0-a238-32e57aca5379 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:45.706919 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e475ff7c-b270-4041-a2d7-35b4b153c209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:45.707608 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e475ff7c-b270-4041-a2d7-35b4b153c209 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:45.707892 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e475ff7c-b270-4041-a2d7-35b4b153c209 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:45.708166 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e475ff7c-b270-4041-a2d7-35b4b153c209 None admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:45.709075 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e475ff7c-b270-4041-a2d7-35b4b153c209 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:45.709390 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e475ff7c-b270-4041-a2d7-35b4b153c209 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:45.710841 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e475ff7c-b270-4041-a2d7-35b4b153c209 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:45.729362 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 518/1050] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Jun 16 17:31:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:31:45.730736 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa162b81-f066-48a0-8c8a-28b6f0f5fb13 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:45.840942 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa162b81-f066-48a0-8c8a-28b6f0f5fb13 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:45.879020 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514bb9d4-5376-4193-beb6-6ae97fa68376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:45.879683 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514bb9d4-5376-4193-beb6-6ae97fa68376 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:45.879963 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514bb9d4-5376-4193-beb6-6ae97fa68376 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:45.880236 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514bb9d4-5376-4193-beb6-6ae97fa68376 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:45.881095 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514bb9d4-5376-4193-beb6-6ae97fa68376 None admin] RBAC: Adding query filter params (role.id=6758650940fd4374ab9e18d56545575e, scope.project.id=8abb87f2f141451a8c6ed832db82be18, user.id=f86c16179a70474a817be9fda33ece9d) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:45.881404 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514bb9d4-5376-4193-beb6-6ae97fa68376 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:45.882829 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514bb9d4-5376-4193-beb6-6ae97fa68376 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:45.896180 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 533/1051] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jun 16 17:31:45 2022] GET /identity/v3/role_assignments?user.id=f86c16179a70474a817be9fda33ece9d&scope.project.id=8abb87f2f141451a8c6ed832db82be18&role.id=6758650940fd4374ab9e18d56545575e => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:47.347009 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ca003d-0fe5-4c73-a04a-38ef33631235 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:47.347526 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ca003d-0fe5-4c73-a04a-38ef33631235 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:47.347646 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ca003d-0fe5-4c73-a04a-38ef33631235 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:47.349121 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 519/1052] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:31:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:31:47.354142 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162a5ac8-a7a6-4f37-be7b-1ab8ad95bcee None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:47.354542 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162a5ac8-a7a6-4f37-be7b-1ab8ad95bcee None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:47.354708 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162a5ac8-a7a6-4f37-be7b-1ab8ad95bcee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:47.371912 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-162a5ac8-a7a6-4f37-be7b-1ab8ad95bcee None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:31:47.373123 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-162a5ac8-a7a6-4f37-be7b-1ab8ad95bcee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:31:47.392195 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:47.446331 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 534/1053] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:31:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:31:47.469502 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ca003d-0fe5-4c73-a04a-38ef33631235 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:47.573925 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ca003d-0fe5-4c73-a04a-38ef33631235 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:47.609301 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d41b029-f378-4539-992c-20ffd145f1cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:47.610026 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d41b029-f378-4539-992c-20ffd145f1cf None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:47.610308 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d41b029-f378-4539-992c-20ffd145f1cf None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:47.610580 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d41b029-f378-4539-992c-20ffd145f1cf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:47.620994 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d41b029-f378-4539-992c-20ffd145f1cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:31:47.628707 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-5d41b029-f378-4539-992c-20ffd145f1cf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:31:47.628707 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:47.628707 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:31:47.628707 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:31:47.628707 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:31:47.628707 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:31:47.628707 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:31:47.628707 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:31:47.628707 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:31:47.628707 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:31:47.628707 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:31:47.628707 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:31:47.628707 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:31:47.628707 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:31:47.628707 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:31:47.628707 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:31:47.629667 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:31:47.629667 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:31:47.629667 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:31:47.629667 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:31:47.629667 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:31:47.629667 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:31:47.629667 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:31:47.629667 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:31:47.629667 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:31:47.629667 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:31:47.629667 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:31:47.629667 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:31:47.629667 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:31:47.629667 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:31:47.629667 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:31:47.629667 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:31:47.630526 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:31:47.630526 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:31:47.630526 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:31:47.630526 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:31:47.630526 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:31:47.630526 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:31:47.630526 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:31:47.630526 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:31:47.630526 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:31:47.630526 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:31:47.630526 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:31:47.630526 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:31:47.632697 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 520/1054] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Jun 16 17:31:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:31:47.635834 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162a5ac8-a7a6-4f37-be7b-1ab8ad95bcee None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:47.738781 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162a5ac8-a7a6-4f37-be7b-1ab8ad95bcee None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:47.777459 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd08bc2-91b6-4912-bf9c-2a6b679390ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:47.778144 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd08bc2-91b6-4912-bf9c-2a6b679390ee None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:47.778431 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd08bc2-91b6-4912-bf9c-2a6b679390ee None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:47.778709 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd08bc2-91b6-4912-bf9c-2a6b679390ee None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:47.779457 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd08bc2-91b6-4912-bf9c-2a6b679390ee None admin] RBAC: Adding query filter params (name=admin) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:47.779767 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd08bc2-91b6-4912-bf9c-2a6b679390ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:47.782180 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd08bc2-91b6-4912-bf9c-2a6b679390ee None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:47.795487 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 535/1055] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jun 16 17:31:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:47.797549 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d41b029-f378-4539-992c-20ffd145f1cf None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:47.809043 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:47.817505 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07325b9-2f2c-4085-b592-5a47654085b3 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:47.817794 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07325b9-2f2c-4085-b592-5a47654085b3 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:47.818078 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07325b9-2f2c-4085-b592-5a47654085b3 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:47.819734 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 536/1056] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:31:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:31:47.824608 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28d1ea2-2151-4b26-9ce9-8c7a5bb0ee8b None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:47.824907 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28d1ea2-2151-4b26-9ce9-8c7a5bb0ee8b None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:47.825191 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28d1ea2-2151-4b26-9ce9-8c7a5bb0ee8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:47.842870 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-c28d1ea2-2151-4b26-9ce9-8c7a5bb0ee8b None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:31:47.844682 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-c28d1ea2-2151-4b26-9ce9-8c7a5bb0ee8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:31:47.854786 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:47.919291 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d41b029-f378-4539-992c-20ffd145f1cf None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:47.923974 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 537/1057] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:31:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 17:31:47.947110 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28d1ea2-2151-4b26-9ce9-8c7a5bb0ee8b None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:47.963296 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94ff3a1-914e-4ae9-a332-f28c9dce5280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:47.964506 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94ff3a1-914e-4ae9-a332-f28c9dce5280 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:47.965360 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94ff3a1-914e-4ae9-a332-f28c9dce5280 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:47.965645 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94ff3a1-914e-4ae9-a332-f28c9dce5280 None admin] PATH_INFO: `/v3/users/1cd284e6dc854345afad006a8807211a` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:47.982242 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94ff3a1-914e-4ae9-a332-f28c9dce5280 None admin] RBAC: Authorizing `identity:get_user(user_id=1cd284e6dc854345afad006a8807211a)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:47.983831 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94ff3a1-914e-4ae9-a332-f28c9dce5280 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:47.985767 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 521/1058] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jun 16 17:31:47 2022] GET /identity/v3/users/1cd284e6dc854345afad006a8807211a => generated 285 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:47.991226 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94ff3a1-914e-4ae9-a332-f28c9dce5280 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:48.054538 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28d1ea2-2151-4b26-9ce9-8c7a5bb0ee8b None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:48.065066 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:48.089290 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b4922a-c8ae-4f25-98c7-6c6597a41fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:48.089946 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b4922a-c8ae-4f25-98c7-6c6597a41fe2 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:48.090229 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b4922a-c8ae-4f25-98c7-6c6597a41fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:48.090507 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b4922a-c8ae-4f25-98c7-6c6597a41fe2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:48.100544 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94ff3a1-914e-4ae9-a332-f28c9dce5280 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:48.105277 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:48.134506 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b91eee-937b-4ddf-8fbd-2a2eb152d4e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:48.135203 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b91eee-937b-4ddf-8fbd-2a2eb152d4e8 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:48.135440 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b91eee-937b-4ddf-8fbd-2a2eb152d4e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:48.135714 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b91eee-937b-4ddf-8fbd-2a2eb152d4e8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:48.160538 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b4922a-c8ae-4f25-98c7-6c6597a41fe2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:48.161989 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b4922a-c8ae-4f25-98c7-6c6597a41fe2 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:48.203366 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b91eee-937b-4ddf-8fbd-2a2eb152d4e8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:48.204868 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b91eee-937b-4ddf-8fbd-2a2eb152d4e8 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:48.261086 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 538/1059] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jun 16 17:31:47 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 16 17:31:48.262251 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ddb6a6-2d24-4254-a76c-e450f56391a3 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:48.262545 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ddb6a6-2d24-4254-a76c-e450f56391a3 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:48.262817 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ddb6a6-2d24-4254-a76c-e450f56391a3 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:48.264231 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 539/1060] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:31:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:31:48.267368 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ddb6a6-2d24-4254-a76c-e450f56391a3 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:48.269353 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:48.308425 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 522/1061] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jun 16 17:31:47 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 16 17:31:48.308425 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd21baa9-148c-4363-90f0-a58917a7968d None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:48.308425 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd21baa9-148c-4363-90f0-a58917a7968d None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:48.308768 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd21baa9-148c-4363-90f0-a58917a7968d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:48.317803 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:48.325700 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-cd21baa9-148c-4363-90f0-a58917a7968d None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:31:48.326898 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-cd21baa9-148c-4363-90f0-a58917a7968d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:31:48.373516 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ddb6a6-2d24-4254-a76c-e450f56391a3 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:48.396529 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 523/1062] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:31:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:31:48.397377 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd21baa9-148c-4363-90f0-a58917a7968d None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:48.414822 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94c633d-4a60-48ba-9c3d-181682fcac56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:48.415674 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94c633d-4a60-48ba-9c3d-181682fcac56 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:48.415930 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94c633d-4a60-48ba-9c3d-181682fcac56 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:48.416204 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94c633d-4a60-48ba-9c3d-181682fcac56 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:48.426968 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94c633d-4a60-48ba-9c3d-181682fcac56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:48.428421 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94c633d-4a60-48ba-9c3d-181682fcac56 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:48.446565 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-e94c633d-4a60-48ba-9c3d-181682fcac56 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:31:48.446565 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:48.446565 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:31:48.446565 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:31:48.446565 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:31:48.446565 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:31:48.446565 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:31:48.446565 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:31:48.446565 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:31:48.446565 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:31:48.446565 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:31:48.446565 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:31:48.446565 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:31:48.446565 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:31:48.446565 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:31:48.447732 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:31:48.447732 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:31:48.447732 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:31:48.447732 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:31:48.447732 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:31:48.447732 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:31:48.447732 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:31:48.447732 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:31:48.447732 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:31:48.447732 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:31:48.447732 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:31:48.447732 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:31:48.447732 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:31:48.447732 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:31:48.447732 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:31:48.447732 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:31:48.447732 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:31:48.448650 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:31:48.448650 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:31:48.448650 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:31:48.448650 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:31:48.448650 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 540/1063] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jun 16 17:31:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:31:48.448650 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94c633d-4a60-48ba-9c3d-181682fcac56 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:48.494693 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:48.501679 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd21baa9-148c-4363-90f0-a58917a7968d None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:48.536647 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5161ba-f78a-40a0-b6ac-2053366c17f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:48.537505 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5161ba-f78a-40a0-b6ac-2053366c17f5 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:48.537790 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5161ba-f78a-40a0-b6ac-2053366c17f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:48.538064 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5161ba-f78a-40a0-b6ac-2053366c17f5 None admin] PATH_INFO: `/v3/projects/ac0783c70b014ada800a364d98afba49` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:48.539236 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5161ba-f78a-40a0-b6ac-2053366c17f5 None admin] RBAC: Authorizing `identity:get_project(project_id=ac0783c70b014ada800a364d98afba49)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:48.540696 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5161ba-f78a-40a0-b6ac-2053366c17f5 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:48.542418 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 524/1064] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jun 16 17:31:48 2022] GET /identity/v3/projects/ac0783c70b014ada800a364d98afba49 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:48.543169 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5161ba-f78a-40a0-b6ac-2053366c17f5 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:48.545716 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:48.553417 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94c633d-4a60-48ba-9c3d-181682fcac56 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:48.587176 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20467d56-be46-43c1-b14d-bc22895c5a98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:48.587984 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20467d56-be46-43c1-b14d-bc22895c5a98 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:48.588263 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20467d56-be46-43c1-b14d-bc22895c5a98 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:48.588568 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20467d56-be46-43c1-b14d-bc22895c5a98 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:48.599165 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20467d56-be46-43c1-b14d-bc22895c5a98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:31:48.607201 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-20467d56-be46-43c1-b14d-bc22895c5a98 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 17:31:48.607201 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:48.607201 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:31:48.607201 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:31:48.607201 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:31:48.607201 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:31:48.607201 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:31:48.607201 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:31:48.607201 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:31:48.607201 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:31:48.607201 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:31:48.607201 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:31:48.607201 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:31:48.607201 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:31:48.607201 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:31:48.607201 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:31:48.608301 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:31:48.608301 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:31:48.608301 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:31:48.608301 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:31:48.608301 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:31:48.608301 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:31:48.608301 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:31:48.608301 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:31:48.608301 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:31:48.608301 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:31:48.608301 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:31:48.608301 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:31:48.608301 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:31:48.608301 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:31:48.608301 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:31:48.608301 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:31:48.609988 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:31:48.609988 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:31:48.609988 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:31:48.609988 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:31:48.609988 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:31:48.609988 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:31:48.609988 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:31:48.609988 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:31:48.609988 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:31:48.609988 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:31:48.609988 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 17:31:48.609988 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:31:48.609988 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 541/1065] 10.222.0.42 () {56 vars in 1120 bytes} [Thu Jun 16 17:31:48 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:31:48.609988 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20467d56-be46-43c1-b14d-bc22895c5a98 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:48.648054 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5161ba-f78a-40a0-b6ac-2053366c17f5 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:48.688944 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5551f25c-8977-4de2-8b00-72e0e2bb4408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:48.689611 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5551f25c-8977-4de2-8b00-72e0e2bb4408 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:48.689896 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5551f25c-8977-4de2-8b00-72e0e2bb4408 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:48.690172 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5551f25c-8977-4de2-8b00-72e0e2bb4408 None admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:48.690899 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5551f25c-8977-4de2-8b00-72e0e2bb4408 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:48.691205 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5551f25c-8977-4de2-8b00-72e0e2bb4408 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:48.692603 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5551f25c-8977-4de2-8b00-72e0e2bb4408 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:48.704860 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 525/1066] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jun 16 17:31:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:48.705748 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5551f25c-8977-4de2-8b00-72e0e2bb4408 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:48.714799 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20467d56-be46-43c1-b14d-bc22895c5a98 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:48.746003 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:48.753247 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1875381b-fea9-41ef-b210-0bce0c4f2eaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:48.753930 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1875381b-fea9-41ef-b210-0bce0c4f2eaa None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:48.754214 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1875381b-fea9-41ef-b210-0bce0c4f2eaa None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:48.754495 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1875381b-fea9-41ef-b210-0bce0c4f2eaa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:48.755344 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1875381b-fea9-41ef-b210-0bce0c4f2eaa None admin] RBAC: Adding query filter params (role.id=3ca2dcbbd29d4a31a3ab78f2299bc239, scope.project.id=ac0783c70b014ada800a364d98afba49, user.id=1cd284e6dc854345afad006a8807211a) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:48.755722 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1875381b-fea9-41ef-b210-0bce0c4f2eaa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:48.757250 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1875381b-fea9-41ef-b210-0bce0c4f2eaa None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:48.763756 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 542/1067] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jun 16 17:31:48 2022] GET /identity/v3/role_assignments?user.id=1cd284e6dc854345afad006a8807211a&scope.project.id=ac0783c70b014ada800a364d98afba49&role.id=3ca2dcbbd29d4a31a3ab78f2299bc239 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:48.848654 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5551f25c-8977-4de2-8b00-72e0e2bb4408 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:48.899751 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbb3c70-5154-45ee-aa9c-da4d24dc709e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:48.900365 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbb3c70-5154-45ee-aa9c-da4d24dc709e None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:48.901231 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbb3c70-5154-45ee-aa9c-da4d24dc709e None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:48.901231 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbb3c70-5154-45ee-aa9c-da4d24dc709e None admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:48.901719 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbb3c70-5154-45ee-aa9c-da4d24dc709e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:48.902013 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbb3c70-5154-45ee-aa9c-da4d24dc709e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:48.903417 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbb3c70-5154-45ee-aa9c-da4d24dc709e None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:48.912948 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 526/1068] 10.222.0.42 () {56 vars in 1115 bytes} [Thu Jun 16 17:31:48 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:48.918776 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1875381b-fea9-41ef-b210-0bce0c4f2eaa None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:48.949678 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:49.049845 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1875381b-fea9-41ef-b210-0bce0c4f2eaa None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:49.083293 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d00ef5c-d0cf-418c-91de-c2b22de6a3b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:49.084077 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d00ef5c-d0cf-418c-91de-c2b22de6a3b4 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:49.084352 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d00ef5c-d0cf-418c-91de-c2b22de6a3b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:49.084647 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d00ef5c-d0cf-418c-91de-c2b22de6a3b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:49.095919 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d00ef5c-d0cf-418c-91de-c2b22de6a3b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:49.097385 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d00ef5c-d0cf-418c-91de-c2b22de6a3b4 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:49.114697 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-7d00ef5c-d0cf-418c-91de-c2b22de6a3b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:31:49.114697 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:49.114697 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:31:49.114697 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:31:49.114697 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:31:49.114697 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:31:49.114697 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:31:49.114697 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:31:49.114697 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:31:49.114697 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:31:49.114697 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:31:49.114697 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:31:49.114697 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:31:49.114697 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:31:49.114697 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:31:49.115757 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:31:49.115757 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:31:49.115757 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:31:49.115757 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:31:49.115757 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:31:49.115757 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:31:49.115757 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:31:49.115757 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:31:49.115757 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:31:49.115757 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:31:49.115757 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:31:49.115757 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:31:49.115757 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:31:49.115757 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:31:49.115757 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:31:49.115757 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:31:49.115757 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:31:49.117016 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:31:49.117016 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:31:49.117016 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:31:49.117016 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:31:49.117016 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 543/1069] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jun 16 17:31:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:31:49.121535 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbb3c70-5154-45ee-aa9c-da4d24dc709e None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:49.205453 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:49.221830 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbb3c70-5154-45ee-aa9c-da4d24dc709e None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:49.255453 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba122da0-2159-45f6-a419-490651f8c330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:49.256121 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba122da0-2159-45f6-a419-490651f8c330 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:49.256432 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba122da0-2159-45f6-a419-490651f8c330 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:49.256713 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba122da0-2159-45f6-a419-490651f8c330 None admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:49.257437 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba122da0-2159-45f6-a419-490651f8c330 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:49.257745 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba122da0-2159-45f6-a419-490651f8c330 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:49.259121 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba122da0-2159-45f6-a419-490651f8c330 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:49.271208 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 527/1070] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jun 16 17:31:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:49.276881 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d00ef5c-d0cf-418c-91de-c2b22de6a3b4 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:49.357611 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:49.397056 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d00ef5c-d0cf-418c-91de-c2b22de6a3b4 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:49.431217 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6f7fb2-4c1c-4e1c-852a-ca15e0ea8bb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:49.432012 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6f7fb2-4c1c-4e1c-852a-ca15e0ea8bb8 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:49.432298 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6f7fb2-4c1c-4e1c-852a-ca15e0ea8bb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:49.432606 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6f7fb2-4c1c-4e1c-852a-ca15e0ea8bb8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:49.440252 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6f7fb2-4c1c-4e1c-852a-ca15e0ea8bb8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:49.441714 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6f7fb2-4c1c-4e1c-852a-ca15e0ea8bb8 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:49.455665 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-5b6f7fb2-4c1c-4e1c-852a-ca15e0ea8bb8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 17:31:49.455665 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:49.455665 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:31:49.455665 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:31:49.455665 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:31:49.455665 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:31:49.455665 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:31:49.455665 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:31:49.455665 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:31:49.455665 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:31:49.455665 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:31:49.455665 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:31:49.455665 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:31:49.455665 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:31:49.455665 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:31:49.455665 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:31:49.456857 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:31:49.456857 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:31:49.456857 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:31:49.456857 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:31:49.456857 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:31:49.456857 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:31:49.456857 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:31:49.456857 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:31:49.456857 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:31:49.456857 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:31:49.456857 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:31:49.456857 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:31:49.456857 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:31:49.456857 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:31:49.456857 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:31:49.456857 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:31:49.456857 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:31:49.456857 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:31:49.458222 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:31:49.458222 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:31:49.458222 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:31:49.458222 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:31:49.458222 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:31:49.458222 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:31:49.458222 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:31:49.458222 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:31:49.458222 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 17:31:49.458222 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:31:49.458222 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 544/1071] 10.222.0.42 () {56 vars in 1093 bytes} [Thu Jun 16 17:31:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:31:49.462504 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba122da0-2159-45f6-a419-490651f8c330 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:49.560166 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba122da0-2159-45f6-a419-490651f8c330 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:49.593627 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89b8b09-0f74-42a6-922d-a3a59e76d727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:49.594285 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89b8b09-0f74-42a6-922d-a3a59e76d727 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:49.594570 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89b8b09-0f74-42a6-922d-a3a59e76d727 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:49.594910 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89b8b09-0f74-42a6-922d-a3a59e76d727 None admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:49.595733 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89b8b09-0f74-42a6-922d-a3a59e76d727 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:49.596037 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89b8b09-0f74-42a6-922d-a3a59e76d727 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:49.597505 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89b8b09-0f74-42a6-922d-a3a59e76d727 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:49.614971 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 528/1072] 10.222.0.42 () {56 vars in 1133 bytes} [Thu Jun 16 17:31:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:49.620499 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6f7fb2-4c1c-4e1c-852a-ca15e0ea8bb8 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:49.720770 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6f7fb2-4c1c-4e1c-852a-ca15e0ea8bb8 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:49.748576 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:49.754383 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55f8652-5a13-4325-bd60-c31f6541d8e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:49.755173 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55f8652-5a13-4325-bd60-c31f6541d8e7 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:49.755458 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55f8652-5a13-4325-bd60-c31f6541d8e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:49.755735 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55f8652-5a13-4325-bd60-c31f6541d8e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:49.767224 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55f8652-5a13-4325-bd60-c31f6541d8e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:49.768800 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55f8652-5a13-4325-bd60-c31f6541d8e7 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:49.786065 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-a55f8652-5a13-4325-bd60-c31f6541d8e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:31:49.786065 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:49.786065 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:31:49.786065 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:31:49.786065 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:31:49.786065 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:31:49.786065 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:31:49.786065 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:31:49.786065 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:31:49.786065 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:31:49.786065 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:31:49.786065 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:31:49.786065 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:31:49.786065 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:31:49.786065 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:31:49.787044 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:31:49.787044 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:31:49.787044 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:31:49.787044 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:31:49.787044 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:31:49.787044 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:31:49.787044 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:31:49.787044 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:31:49.787044 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:31:49.787044 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:31:49.787044 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:31:49.787044 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:31:49.787044 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:31:49.787044 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:31:49.787044 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:31:49.787044 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:31:49.787044 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:31:49.787943 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:31:49.787943 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:31:49.787943 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:31:49.787943 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:31:49.787943 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 545/1073] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jun 16 17:31:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:31:49.792931 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89b8b09-0f74-42a6-922d-a3a59e76d727 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:49.797419 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:49.893297 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89b8b09-0f74-42a6-922d-a3a59e76d727 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:49.928828 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71eb90ad-cb2c-4744-8db8-3123edd48529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:49.929492 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eb90ad-cb2c-4744-8db8-3123edd48529 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:49.929776 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eb90ad-cb2c-4744-8db8-3123edd48529 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:49.930051 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eb90ad-cb2c-4744-8db8-3123edd48529 None admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:49.930766 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71eb90ad-cb2c-4744-8db8-3123edd48529 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:49.931072 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71eb90ad-cb2c-4744-8db8-3123edd48529 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:49.932457 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71eb90ad-cb2c-4744-8db8-3123edd48529 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:49.943852 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 529/1074] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jun 16 17:31:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:49.949442 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55f8652-5a13-4325-bd60-c31f6541d8e7 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:50.051476 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55f8652-5a13-4325-bd60-c31f6541d8e7 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:50.084756 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b5f132-694d-44c5-93b8-e3864fcfd4e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:50.085552 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b5f132-694d-44c5-93b8-e3864fcfd4e3 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:50.085835 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b5f132-694d-44c5-93b8-e3864fcfd4e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:50.086116 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b5f132-694d-44c5-93b8-e3864fcfd4e3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:50.092692 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b5f132-694d-44c5-93b8-e3864fcfd4e3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:50.094187 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b5f132-694d-44c5-93b8-e3864fcfd4e3 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:50.113256 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-16b5f132-694d-44c5-93b8-e3864fcfd4e3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:31:50.113256 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:50.113256 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:31:50.113256 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:31:50.113256 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:31:50.113256 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:31:50.113256 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:31:50.113256 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:31:50.113256 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:31:50.113256 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:31:50.113256 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:31:50.113256 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:31:50.113256 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:31:50.113256 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:31:50.113256 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:31:50.114162 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:31:50.114162 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:31:50.114162 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:31:50.114162 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:31:50.114162 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:31:50.114162 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:31:50.114162 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:31:50.114162 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:31:50.114162 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:31:50.114162 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:31:50.114162 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:31:50.114162 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:31:50.114162 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:31:50.114162 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:31:50.114162 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:31:50.114162 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:31:50.114162 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:31:50.115227 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:31:50.115227 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:31:50.115227 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:31:50.115227 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:31:50.115227 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:31:50.115227 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:31:50.115227 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:31:50.115227 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:31:50.115227 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 546/1075] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jun 16 17:31:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:31:50.120239 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71eb90ad-cb2c-4744-8db8-3123edd48529 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:50.182093 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:50.220649 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71eb90ad-cb2c-4744-8db8-3123edd48529 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:50.253546 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a483995b-4bb4-4d44-8ed5-aa86c0e97ba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:50.254204 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a483995b-4bb4-4d44-8ed5-aa86c0e97ba6 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:50.254493 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a483995b-4bb4-4d44-8ed5-aa86c0e97ba6 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:50.254774 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a483995b-4bb4-4d44-8ed5-aa86c0e97ba6 None admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:50.255560 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a483995b-4bb4-4d44-8ed5-aa86c0e97ba6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:50.255873 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a483995b-4bb4-4d44-8ed5-aa86c0e97ba6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:50.257335 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a483995b-4bb4-4d44-8ed5-aa86c0e97ba6 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:50.270044 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 530/1076] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Jun 16 17:31:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:50.275874 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b5f132-694d-44c5-93b8-e3864fcfd4e3 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:50.320930 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:50.375695 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b5f132-694d-44c5-93b8-e3864fcfd4e3 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:50.409242 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bb966a-3251-43ff-b533-29be5b4399b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:50.410128 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bb966a-3251-43ff-b533-29be5b4399b6 None admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:50.410413 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bb966a-3251-43ff-b533-29be5b4399b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:50.410690 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bb966a-3251-43ff-b533-29be5b4399b6 None admin] PATH_INFO: `/v3/projects/8abb87f2f141451a8c6ed832db82be18/users/f86c16179a70474a817be9fda33ece9d/roles/6758650940fd4374ab9e18d56545575e` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:50.412920 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bb966a-3251-43ff-b533-29be5b4399b6 None admin] RBAC: Authorizing `identity:create_grant(project_id=8abb87f2f141451a8c6ed832db82be18, user_id=f86c16179a70474a817be9fda33ece9d, role_id=6758650940fd4374ab9e18d56545575e)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:50.414353 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bb966a-3251-43ff-b533-29be5b4399b6 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:50.427584 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 547/1077] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Jun 16 17:31:50 2022] PUT /identity/v3/projects/8abb87f2f141451a8c6ed832db82be18/users/f86c16179a70474a817be9fda33ece9d/roles/6758650940fd4374ab9e18d56545575e => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:31:50.987977 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49f1d2b-2eb1-47ee-9265-3ec34ea1a74d None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:50.988518 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49f1d2b-2eb1-47ee-9265-3ec34ea1a74d None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:50.988604 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49f1d2b-2eb1-47ee-9265-3ec34ea1a74d None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:50.989940 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 531/1078] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:31:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:31:50.995276 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a05bc18-28b0-4ae4-8f09-c1413644f000 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:50.995566 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a05bc18-28b0-4ae4-8f09-c1413644f000 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:50.995843 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a05bc18-28b0-4ae4-8f09-c1413644f000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:51.013323 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-1a05bc18-28b0-4ae4-8f09-c1413644f000 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:31:51.014501 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-1a05bc18-28b0-4ae4-8f09-c1413644f000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:31:51.082624 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 548/1079] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:31:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:31:51.105676 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49f1d2b-2eb1-47ee-9265-3ec34ea1a74d None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:51.210680 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49f1d2b-2eb1-47ee-9265-3ec34ea1a74d None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:51.234789 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:51.245638 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5e256f-19e6-4299-ba07-d7b1959b6017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:51.246426 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5e256f-19e6-4299-ba07-d7b1959b6017 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:51.246709 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5e256f-19e6-4299-ba07-d7b1959b6017 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:51.246983 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5e256f-19e6-4299-ba07-d7b1959b6017 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:51.257535 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5e256f-19e6-4299-ba07-d7b1959b6017 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:31:51.265234 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-2c5e256f-19e6-4299-ba07-d7b1959b6017 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:31:51.265234 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:51.265234 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:31:51.265234 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:31:51.265234 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:31:51.265234 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:31:51.265234 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:31:51.265234 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:31:51.265234 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:31:51.265234 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:31:51.265234 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:31:51.265234 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:31:51.265234 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:31:51.265234 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:31:51.265234 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:31:51.265234 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:31:51.266577 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:31:51.266577 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:31:51.266577 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:31:51.266577 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:31:51.266577 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:31:51.266577 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:31:51.266577 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:31:51.266577 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:31:51.266577 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:31:51.266577 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:31:51.266577 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:31:51.266577 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:31:51.266577 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:31:51.266577 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:31:51.266577 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:31:51.266577 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:31:51.267859 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:31:51.267859 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:31:51.267859 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:31:51.267859 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:31:51.267859 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:31:51.267859 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:31:51.267859 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:31:51.267859 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:31:51.267859 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:31:51.267859 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:31:51.267859 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:31:51.267859 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:31:51.267859 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 532/1080] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Jun 16 17:31:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:31:51.272894 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a05bc18-28b0-4ae4-8f09-c1413644f000 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:51.305751 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:51.372291 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a05bc18-28b0-4ae4-8f09-c1413644f000 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:51.405727 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aca751f-2ba5-47af-bf01-cfae2c7f59d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:51.406400 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aca751f-2ba5-47af-bf01-cfae2c7f59d5 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:51.406685 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aca751f-2ba5-47af-bf01-cfae2c7f59d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:51.406963 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aca751f-2ba5-47af-bf01-cfae2c7f59d5 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:51.407698 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aca751f-2ba5-47af-bf01-cfae2c7f59d5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:51.408008 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aca751f-2ba5-47af-bf01-cfae2c7f59d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:51.409407 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aca751f-2ba5-47af-bf01-cfae2c7f59d5 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:51.417350 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 549/1081] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jun 16 17:31:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:51.423802 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5e256f-19e6-4299-ba07-d7b1959b6017 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:51.524163 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5e256f-19e6-4299-ba07-d7b1959b6017 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:51.558538 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff0051b-234d-4361-a3f5-6cec955e8dae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:51.559412 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff0051b-234d-4361-a3f5-6cec955e8dae None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:51.559696 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff0051b-234d-4361-a3f5-6cec955e8dae None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:51.559974 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff0051b-234d-4361-a3f5-6cec955e8dae None admin] PATH_INFO: `/v3/users/1cd284e6dc854345afad006a8807211a` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:51.561228 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff0051b-234d-4361-a3f5-6cec955e8dae None admin] RBAC: Authorizing `identity:get_user(user_id=1cd284e6dc854345afad006a8807211a)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:51.562840 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff0051b-234d-4361-a3f5-6cec955e8dae None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:51.564506 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 533/1082] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jun 16 17:31:51 2022] GET /identity/v3/users/1cd284e6dc854345afad006a8807211a => generated 285 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:51.570214 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aca751f-2ba5-47af-bf01-cfae2c7f59d5 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:51.663872 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:51.670318 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aca751f-2ba5-47af-bf01-cfae2c7f59d5 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:51.705205 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c7facc-0eec-4c87-a1a8-e6c26ceb289d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:51.706062 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c7facc-0eec-4c87-a1a8-e6c26ceb289d None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:51.706347 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c7facc-0eec-4c87-a1a8-e6c26ceb289d None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:51.706624 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c7facc-0eec-4c87-a1a8-e6c26ceb289d None admin] PATH_INFO: `/v3/projects/ac0783c70b014ada800a364d98afba49` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:51.707806 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c7facc-0eec-4c87-a1a8-e6c26ceb289d None admin] RBAC: Authorizing `identity:get_project(project_id=ac0783c70b014ada800a364d98afba49)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:51.709277 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c7facc-0eec-4c87-a1a8-e6c26ceb289d None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:51.710969 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 550/1083] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jun 16 17:31:51 2022] GET /identity/v3/projects/ac0783c70b014ada800a364d98afba49 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:51.716878 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff0051b-234d-4361-a3f5-6cec955e8dae None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:51.769976 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:51.817921 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff0051b-234d-4361-a3f5-6cec955e8dae None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:51.850555 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e2cd85-5977-4318-a49f-a116cb54e633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:51.851433 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e2cd85-5977-4318-a49f-a116cb54e633 None admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:51.851714 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e2cd85-5977-4318-a49f-a116cb54e633 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:51.851988 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e2cd85-5977-4318-a49f-a116cb54e633 None admin] PATH_INFO: `/v3/projects/ac0783c70b014ada800a364d98afba49/users/1cd284e6dc854345afad006a8807211a/roles/3ca2dcbbd29d4a31a3ab78f2299bc239` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:51.854027 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e2cd85-5977-4318-a49f-a116cb54e633 None admin] RBAC: Authorizing `identity:create_grant(project_id=ac0783c70b014ada800a364d98afba49, user_id=1cd284e6dc854345afad006a8807211a, role_id=3ca2dcbbd29d4a31a3ab78f2299bc239)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:51.855477 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e2cd85-5977-4318-a49f-a116cb54e633 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:51.870971 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 534/1084] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Jun 16 17:31:51 2022] PUT /identity/v3/projects/ac0783c70b014ada800a364d98afba49/users/1cd284e6dc854345afad006a8807211a/roles/3ca2dcbbd29d4a31a3ab78f2299bc239 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:31:52.536935 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57f5294-dcb3-4a74-aa27-fa9aff89066f None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:52.537461 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57f5294-dcb3-4a74-aa27-fa9aff89066f None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:52.537583 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57f5294-dcb3-4a74-aa27-fa9aff89066f None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:52.539058 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 551/1085] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:31:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:31:52.544290 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f632755-79f8-4253-9602-0a854e3361aa None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:52.544775 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f632755-79f8-4253-9602-0a854e3361aa None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:52.545025 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f632755-79f8-4253-9602-0a854e3361aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:52.563347 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-1f632755-79f8-4253-9602-0a854e3361aa None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:31:52.564582 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-1f632755-79f8-4253-9602-0a854e3361aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:31:52.631513 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 535/1086] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:31:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:31:52.656649 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57f5294-dcb3-4a74-aa27-fa9aff89066f None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:52.760168 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57f5294-dcb3-4a74-aa27-fa9aff89066f None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:52.798801 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e05d46a-0527-4f48-a2fc-1fd239a56b42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:52.799528 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e05d46a-0527-4f48-a2fc-1fd239a56b42 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:52.799814 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e05d46a-0527-4f48-a2fc-1fd239a56b42 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:52.800090 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e05d46a-0527-4f48-a2fc-1fd239a56b42 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:52.810146 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e05d46a-0527-4f48-a2fc-1fd239a56b42 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:31:52.818383 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-0e05d46a-0527-4f48-a2fc-1fd239a56b42 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 17:31:52.818383 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:52.818383 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:31:52.818383 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:31:52.818383 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:31:52.818383 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:31:52.818383 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:31:52.818383 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:31:52.818383 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:31:52.818383 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:31:52.818383 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:31:52.818383 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:31:52.818383 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:31:52.818383 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:31:52.818383 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:31:52.818383 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:31:52.819412 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:31:52.819412 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:31:52.819412 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:31:52.819412 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:31:52.819412 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:31:52.819412 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:31:52.819412 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:31:52.819412 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:31:52.819412 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:31:52.819412 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:31:52.819412 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:31:52.819412 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:31:52.819412 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:31:52.819412 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:31:52.819412 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:31:52.819412 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:31:52.820294 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:31:52.820294 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:31:52.820294 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:31:52.820294 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:31:52.820294 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:31:52.820294 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:31:52.820294 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:31:52.820294 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:31:52.820294 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:31:52.820294 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:31:52.820294 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 17:31:52.820294 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:31:52.820294 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 552/1087] 10.222.0.42 () {56 vars in 1120 bytes} [Thu Jun 16 17:31:52 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:31:52.825518 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f632755-79f8-4253-9602-0a854e3361aa None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:52.868908 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:52.926404 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f632755-79f8-4253-9602-0a854e3361aa None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:52.959117 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b025dc-5e3a-41b1-802f-7c9196a8e0d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:52.959774 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b025dc-5e3a-41b1-802f-7c9196a8e0d4 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:52.960053 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b025dc-5e3a-41b1-802f-7c9196a8e0d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:52.960326 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b025dc-5e3a-41b1-802f-7c9196a8e0d4 None admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:52.961091 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b025dc-5e3a-41b1-802f-7c9196a8e0d4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:52.961393 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b025dc-5e3a-41b1-802f-7c9196a8e0d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:52.962754 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b025dc-5e3a-41b1-802f-7c9196a8e0d4 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:52.971207 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 536/1088] 10.222.0.42 () {56 vars in 1115 bytes} [Thu Jun 16 17:31:52 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:52.976985 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e05d46a-0527-4f48-a2fc-1fd239a56b42 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:53.004477 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:53.079315 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e05d46a-0527-4f48-a2fc-1fd239a56b42 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:53.116912 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50f7826-dd5b-46d2-9fa4-32d42503eea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:53.118266 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50f7826-dd5b-46d2-9fa4-32d42503eea8 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:53.118795 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50f7826-dd5b-46d2-9fa4-32d42503eea8 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:53.119312 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50f7826-dd5b-46d2-9fa4-32d42503eea8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:53.132524 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50f7826-dd5b-46d2-9fa4-32d42503eea8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:53.133974 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50f7826-dd5b-46d2-9fa4-32d42503eea8 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:53.151640 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-b50f7826-dd5b-46d2-9fa4-32d42503eea8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:31:53.151640 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:53.151640 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:31:53.151640 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:31:53.151640 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:31:53.151640 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:31:53.151640 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:31:53.151640 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:31:53.151640 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:31:53.151640 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:31:53.151640 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:31:53.151640 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:31:53.151640 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:31:53.151640 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:31:53.151640 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:31:53.152754 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:31:53.152754 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:31:53.152754 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:31:53.152754 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:31:53.152754 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:31:53.152754 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:31:53.152754 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:31:53.152754 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:31:53.152754 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:31:53.152754 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:31:53.152754 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:31:53.152754 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:31:53.152754 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:31:53.152754 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:31:53.152754 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:31:53.152754 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:31:53.152754 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:31:53.153659 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:31:53.153659 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:31:53.153659 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:31:53.153659 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:31:53.153659 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 553/1089] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jun 16 17:31:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:31:53.159313 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b025dc-5e3a-41b1-802f-7c9196a8e0d4 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:53.258942 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b025dc-5e3a-41b1-802f-7c9196a8e0d4 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:53.261288 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:53.293270 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc31c273-0dff-4184-9c4b-d893f8d4823a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:53.293926 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc31c273-0dff-4184-9c4b-d893f8d4823a None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:53.294210 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc31c273-0dff-4184-9c4b-d893f8d4823a None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:53.294485 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc31c273-0dff-4184-9c4b-d893f8d4823a None admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:53.295200 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc31c273-0dff-4184-9c4b-d893f8d4823a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:53.295506 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc31c273-0dff-4184-9c4b-d893f8d4823a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:53.297976 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc31c273-0dff-4184-9c4b-d893f8d4823a None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:53.309982 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 537/1090] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jun 16 17:31:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:53.316067 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50f7826-dd5b-46d2-9fa4-32d42503eea8 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:53.393873 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:53.416907 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50f7826-dd5b-46d2-9fa4-32d42503eea8 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:53.448859 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bb1e88-fd69-4d4e-99c1-de4aff887a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:53.449639 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bb1e88-fd69-4d4e-99c1-de4aff887a4b None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:53.449918 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bb1e88-fd69-4d4e-99c1-de4aff887a4b None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:53.450255 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bb1e88-fd69-4d4e-99c1-de4aff887a4b None admin] PATH_INFO: `/v3/users/nova` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:53.458424 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bb1e88-fd69-4d4e-99c1-de4aff887a4b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:53.459846 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bb1e88-fd69-4d4e-99c1-de4aff887a4b None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:53.480273 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-a5bb1e88-fd69-4d4e-99c1-de4aff887a4b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 17:31:53.480273 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:53.480273 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:31:53.480273 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:31:53.480273 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:31:53.480273 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:31:53.480273 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:31:53.480273 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:31:53.480273 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:31:53.480273 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:31:53.480273 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:31:53.480273 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:31:53.480273 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:31:53.480273 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:31:53.480273 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:31:53.480273 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:31:53.481277 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:31:53.481277 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:31:53.481277 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:31:53.481277 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:31:53.481277 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:31:53.481277 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:31:53.481277 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:31:53.481277 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:31:53.481277 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:31:53.481277 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:31:53.481277 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:31:53.481277 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:31:53.481277 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:31:53.481277 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:31:53.481277 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:31:53.481277 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:31:53.481277 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:31:53.481277 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:31:53.482257 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:31:53.482257 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:31:53.482257 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:31:53.482257 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:31:53.482257 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:31:53.482257 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:31:53.482257 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:31:53.482257 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:31:53.482257 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 17:31:53.482257 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:31:53.482805 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 554/1091] 10.222.0.42 () {56 vars in 1093 bytes} [Thu Jun 16 17:31:53 2022] GET /identity/v3/users/nova => generated 82 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:31:53.488126 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc31c273-0dff-4184-9c4b-d893f8d4823a None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:53.605816 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc31c273-0dff-4184-9c4b-d893f8d4823a None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:53.639040 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a55b67-e207-4544-b7c2-a5a0cc563ead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:53.640250 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a55b67-e207-4544-b7c2-a5a0cc563ead None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:53.640565 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a55b67-e207-4544-b7c2-a5a0cc563ead None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:53.640843 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a55b67-e207-4544-b7c2-a5a0cc563ead None admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:53.641930 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a55b67-e207-4544-b7c2-a5a0cc563ead None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:53.642238 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a55b67-e207-4544-b7c2-a5a0cc563ead None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:53.643666 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a55b67-e207-4544-b7c2-a5a0cc563ead None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:53.661132 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 538/1092] 10.222.0.42 () {56 vars in 1133 bytes} [Thu Jun 16 17:31:53 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:31:53.666686 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bb1e88-fd69-4d4e-99c1-de4aff887a4b None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:53.767021 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bb1e88-fd69-4d4e-99c1-de4aff887a4b None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:53.800540 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6808b342-2442-424a-b7d6-74876e886910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:53.801187 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6808b342-2442-424a-b7d6-74876e886910 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:53.801471 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6808b342-2442-424a-b7d6-74876e886910 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:53.801747 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6808b342-2442-424a-b7d6-74876e886910 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:53.805187 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:53.867758 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6808b342-2442-424a-b7d6-74876e886910 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:53.869306 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6808b342-2442-424a-b7d6-74876e886910 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:53.973851 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 555/1093] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jun 16 17:31:53 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 16 17:31:53.979877 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a55b67-e207-4544-b7c2-a5a0cc563ead None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:54.006712 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:54.082089 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a55b67-e207-4544-b7c2-a5a0cc563ead None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:54.116432 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c1ab85-c477-441f-bb3c-d96b09b453bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:54.117320 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c1ab85-c477-441f-bb3c-d96b09b453bd None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:54.117601 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c1ab85-c477-441f-bb3c-d96b09b453bd None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:54.117874 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c1ab85-c477-441f-bb3c-d96b09b453bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:54.129944 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c1ab85-c477-441f-bb3c-d96b09b453bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:54.131426 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c1ab85-c477-441f-bb3c-d96b09b453bd None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:54.150967 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-63c1ab85-c477-441f-bb3c-d96b09b453bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:31:54.150967 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:54.150967 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:31:54.150967 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:31:54.150967 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:31:54.150967 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:31:54.150967 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:31:54.150967 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:31:54.150967 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:31:54.150967 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:31:54.150967 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:31:54.150967 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:31:54.150967 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:31:54.150967 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:31:54.150967 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:31:54.151958 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:31:54.151958 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:31:54.151958 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:31:54.151958 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:31:54.151958 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:31:54.151958 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:31:54.151958 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:31:54.151958 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:31:54.151958 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:31:54.151958 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:31:54.151958 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:31:54.151958 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:31:54.151958 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:31:54.151958 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:31:54.151958 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:31:54.151958 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:31:54.151958 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:31:54.152917 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:31:54.152917 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:31:54.152917 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:31:54.152917 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:31:54.152917 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 539/1094] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jun 16 17:31:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:31:54.158126 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6808b342-2442-424a-b7d6-74876e886910 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:54.176365 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5b9c36-e9a2-484a-8c9f-bc4b9c9e1b42 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:54.176981 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5b9c36-e9a2-484a-8c9f-bc4b9c9e1b42 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:54.177257 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5b9c36-e9a2-484a-8c9f-bc4b9c9e1b42 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:54.178619 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 540/1095] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:31:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:31:54.183399 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce74753-1455-4b23-9d0c-7343026434b4 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:54.183678 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce74753-1455-4b23-9d0c-7343026434b4 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:54.183944 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce74753-1455-4b23-9d0c-7343026434b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:54.194497 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:54.201418 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-1ce74753-1455-4b23-9d0c-7343026434b4 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:31:54.202748 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-1ce74753-1455-4b23-9d0c-7343026434b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:31:54.260107 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6808b342-2442-424a-b7d6-74876e886910 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:54.270278 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 541/1096] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:31:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 17:31:54.294902 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7cf945-dc24-4bef-b30e-73e68804e389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:54.295586 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7cf945-dc24-4bef-b30e-73e68804e389 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:54.296409 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7cf945-dc24-4bef-b30e-73e68804e389 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:54.296758 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7cf945-dc24-4bef-b30e-73e68804e389 None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:54.297494 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7cf945-dc24-4bef-b30e-73e68804e389 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:54.297799 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7cf945-dc24-4bef-b30e-73e68804e389 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:54.299288 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7cf945-dc24-4bef-b30e-73e68804e389 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:54.299406 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce74753-1455-4b23-9d0c-7343026434b4 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:54.302670 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:54.317402 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 556/1097] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jun 16 17:31:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:54.323024 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7cf945-dc24-4bef-b30e-73e68804e389 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:54.403950 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce74753-1455-4b23-9d0c-7343026434b4 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:54.418316 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:54.425343 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7cf945-dc24-4bef-b30e-73e68804e389 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:54.439389 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c76e6c-29f7-4f4a-9f78-033395be8c56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:54.440180 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c76e6c-29f7-4f4a-9f78-033395be8c56 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:54.440498 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c76e6c-29f7-4f4a-9f78-033395be8c56 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:54.440773 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c76e6c-29f7-4f4a-9f78-033395be8c56 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:54.450798 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c76e6c-29f7-4f4a-9f78-033395be8c56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:31:54.458349 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-80c76e6c-29f7-4f4a-9f78-033395be8c56 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:31:54.458349 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:54.458349 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:31:54.458349 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:31:54.458349 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:31:54.458349 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:31:54.458349 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:31:54.458349 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:31:54.458349 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:31:54.458349 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:31:54.458349 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:31:54.458349 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:31:54.458349 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:31:54.458349 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:31:54.458349 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:31:54.458349 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:31:54.459312 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:31:54.459312 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:31:54.459312 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:31:54.459312 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:31:54.459312 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:31:54.459312 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:31:54.459312 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:31:54.459312 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:31:54.459312 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:31:54.459312 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:31:54.459312 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:31:54.459312 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:31:54.459312 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:31:54.459312 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:31:54.459312 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:31:54.459312 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:31:54.460345 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:31:54.460345 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:31:54.460345 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:31:54.460345 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:31:54.460345 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:31:54.460345 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:31:54.460345 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:31:54.460345 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:31:54.460345 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:31:54.460345 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:31:54.460345 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:31:54.460345 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:31:54.461200 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf914f72-b133-45b1-b987-858ccc5fd21b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:54.461200 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf914f72-b133-45b1-b987-858ccc5fd21b None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:54.461361 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 542/1098] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Jun 16 17:31:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:31:54.461432 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf914f72-b133-45b1-b987-858ccc5fd21b None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:54.461432 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf914f72-b133-45b1-b987-858ccc5fd21b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:54.465423 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c76e6c-29f7-4f4a-9f78-033395be8c56 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:54.466812 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf914f72-b133-45b1-b987-858ccc5fd21b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:54.468399 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf914f72-b133-45b1-b987-858ccc5fd21b None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:54.483208 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-bf914f72-b133-45b1-b987-858ccc5fd21b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:31:54.483208 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:31:54.483208 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:31:54.483208 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:31:54.483208 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:31:54.483208 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:31:54.483208 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:31:54.483208 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:31:54.483208 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:31:54.483208 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:31:54.483208 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:31:54.483208 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:31:54.483208 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:31:54.483208 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:31:54.483208 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:31:54.484885 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:31:54.484885 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:31:54.484885 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:31:54.484885 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:31:54.484885 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:31:54.484885 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:31:54.484885 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:31:54.484885 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:31:54.484885 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:31:54.484885 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:31:54.484885 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:31:54.484885 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:31:54.484885 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:31:54.484885 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:31:54.484885 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:31:54.484885 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:31:54.484885 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:31:54.486279 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:31:54.486279 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:31:54.486279 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:31:54.486279 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:31:54.486279 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:31:54.486279 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:31:54.486279 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:31:54.486279 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:31:54.486279 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 557/1099] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jun 16 17:31:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:31:54.490761 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf914f72-b133-45b1-b987-858ccc5fd21b None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:54.528413 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:54.569699 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c76e6c-29f7-4f4a-9f78-033395be8c56 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:54.594315 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf914f72-b133-45b1-b987-858ccc5fd21b None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:54.604357 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc70a82-3c84-4950-b4ca-5065b129ca16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:54.605036 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc70a82-3c84-4950-b4ca-5065b129ca16 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:54.605314 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc70a82-3c84-4950-b4ca-5065b129ca16 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:54.605585 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc70a82-3c84-4950-b4ca-5065b129ca16 None admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:54.606311 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc70a82-3c84-4950-b4ca-5065b129ca16 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:54.606843 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc70a82-3c84-4950-b4ca-5065b129ca16 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:54.607957 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc70a82-3c84-4950-b4ca-5065b129ca16 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:54.616674 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 543/1100] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jun 16 17:31:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:31:54.622376 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc70a82-3c84-4950-b4ca-5065b129ca16 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:54.633728 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c73442-6772-4e91-8343-32549cfae11d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:54.634384 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c73442-6772-4e91-8343-32549cfae11d None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:54.634665 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c73442-6772-4e91-8343-32549cfae11d None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:54.634938 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c73442-6772-4e91-8343-32549cfae11d None admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:54.635706 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c73442-6772-4e91-8343-32549cfae11d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:54.636011 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c73442-6772-4e91-8343-32549cfae11d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:54.637459 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c73442-6772-4e91-8343-32549cfae11d None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:54.649183 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 558/1101] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Jun 16 17:31:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:31:54.655058 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c73442-6772-4e91-8343-32549cfae11d None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:54.673967 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:54.724802 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc70a82-3c84-4950-b4ca-5065b129ca16 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:54.754046 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:54.761754 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c73442-6772-4e91-8343-32549cfae11d None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:54.763115 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4bb2e4-7b21-4e41-b778-869c0531b51c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:54.763983 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4bb2e4-7b21-4e41-b778-869c0531b51c None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:54.764259 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4bb2e4-7b21-4e41-b778-869c0531b51c None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:54.764563 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4bb2e4-7b21-4e41-b778-869c0531b51c None admin] PATH_INFO: `/v3/users/1cd284e6dc854345afad006a8807211a` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:54.765926 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4bb2e4-7b21-4e41-b778-869c0531b51c None admin] RBAC: Authorizing `identity:get_user(user_id=1cd284e6dc854345afad006a8807211a)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:54.767614 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4bb2e4-7b21-4e41-b778-869c0531b51c None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:54.769288 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 544/1102] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jun 16 17:31:54 2022] GET /identity/v3/users/1cd284e6dc854345afad006a8807211a => generated 285 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:31:54.775017 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4bb2e4-7b21-4e41-b778-869c0531b51c None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:54.805400 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02d5cf4-9980-42b3-85da-baecfdc16410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:54.805989 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02d5cf4-9980-42b3-85da-baecfdc16410 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:54.806276 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02d5cf4-9980-42b3-85da-baecfdc16410 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:54.806559 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02d5cf4-9980-42b3-85da-baecfdc16410 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:54.807383 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02d5cf4-9980-42b3-85da-baecfdc16410 None admin] RBAC: Adding query filter params (role.id=6758650940fd4374ab9e18d56545575e, scope.project.id=8abb87f2f141451a8c6ed832db82be18, user.id=f86c16179a70474a817be9fda33ece9d) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:54.807698 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02d5cf4-9980-42b3-85da-baecfdc16410 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:54.809175 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02d5cf4-9980-42b3-85da-baecfdc16410 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:54.815649 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 559/1103] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jun 16 17:31:54 2022] GET /identity/v3/role_assignments?user.id=f86c16179a70474a817be9fda33ece9d&scope.project.id=8abb87f2f141451a8c6ed832db82be18&role.id=6758650940fd4374ab9e18d56545575e => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:31:54.881121 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4bb2e4-7b21-4e41-b778-869c0531b51c None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:54.919416 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04e64a0-e75a-4b49-b427-89a6fd4d709b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:54.920080 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04e64a0-e75a-4b49-b427-89a6fd4d709b None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:54.920364 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04e64a0-e75a-4b49-b427-89a6fd4d709b None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:54.920667 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04e64a0-e75a-4b49-b427-89a6fd4d709b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:54.988869 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:54.996628 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04e64a0-e75a-4b49-b427-89a6fd4d709b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:54.998075 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04e64a0-e75a-4b49-b427-89a6fd4d709b None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:55.100479 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 545/1104] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jun 16 17:31:54 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 16 17:31:55.106573 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02d5cf4-9980-42b3-85da-baecfdc16410 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:55.190209 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:55.211256 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02d5cf4-9980-42b3-85da-baecfdc16410 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:55.245430 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad222d77-0c59-4ccb-8694-40c8ab82191d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:55.246290 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad222d77-0c59-4ccb-8694-40c8ab82191d None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:55.246571 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad222d77-0c59-4ccb-8694-40c8ab82191d None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:55.246848 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad222d77-0c59-4ccb-8694-40c8ab82191d None admin] PATH_INFO: `/v3/projects/ac0783c70b014ada800a364d98afba49` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:55.248060 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad222d77-0c59-4ccb-8694-40c8ab82191d None admin] RBAC: Authorizing `identity:get_project(project_id=ac0783c70b014ada800a364d98afba49)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:55.249779 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad222d77-0c59-4ccb-8694-40c8ab82191d None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:55.252130 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 560/1105] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jun 16 17:31:55 2022] GET /identity/v3/projects/ac0783c70b014ada800a364d98afba49 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:55.258309 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04e64a0-e75a-4b49-b427-89a6fd4d709b None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:55.362251 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:55.369640 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04e64a0-e75a-4b49-b427-89a6fd4d709b None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:55.405878 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1516183-91d7-4ab6-9a1f-e510dc04ee7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:55.406623 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1516183-91d7-4ab6-9a1f-e510dc04ee7f None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:55.406811 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1516183-91d7-4ab6-9a1f-e510dc04ee7f None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:55.407088 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1516183-91d7-4ab6-9a1f-e510dc04ee7f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:55.407917 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1516183-91d7-4ab6-9a1f-e510dc04ee7f None admin] RBAC: Adding query filter params (role.id=3ca2dcbbd29d4a31a3ab78f2299bc239, scope.project.id=ac0783c70b014ada800a364d98afba49, user.id=1cd284e6dc854345afad006a8807211a) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:55.408233 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1516183-91d7-4ab6-9a1f-e510dc04ee7f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:55.409716 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1516183-91d7-4ab6-9a1f-e510dc04ee7f None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:55.415949 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 546/1106] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jun 16 17:31:55 2022] GET /identity/v3/role_assignments?user.id=1cd284e6dc854345afad006a8807211a&scope.project.id=ac0783c70b014ada800a364d98afba49&role.id=3ca2dcbbd29d4a31a3ab78f2299bc239 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:57.618456 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cff0c4-1405-40fd-81f0-cb9f181d8190 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:57.618995 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cff0c4-1405-40fd-81f0-cb9f181d8190 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:57.618995 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cff0c4-1405-40fd-81f0-cb9f181d8190 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:57.620396 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 561/1107] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:31:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:31:57.625457 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6b5b5e-bcb0-4918-bdd4-1df5fc8fd8ed None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:57.625910 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6b5b5e-bcb0-4918-bdd4-1df5fc8fd8ed None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:57.626111 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6b5b5e-bcb0-4918-bdd4-1df5fc8fd8ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:57.642852 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-9b6b5b5e-bcb0-4918-bdd4-1df5fc8fd8ed None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:31:57.644035 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-9b6b5b5e-bcb0-4918-bdd4-1df5fc8fd8ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:31:57.708123 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 547/1108] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:31:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:31:57.734207 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cff0c4-1405-40fd-81f0-cb9f181d8190 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:57.835109 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cff0c4-1405-40fd-81f0-cb9f181d8190 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:57.869726 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c257e2-456b-4b06-8b4f-dee3df2cc61f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:57.870425 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c257e2-456b-4b06-8b4f-dee3df2cc61f None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:57.870710 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c257e2-456b-4b06-8b4f-dee3df2cc61f None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:57.870988 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c257e2-456b-4b06-8b4f-dee3df2cc61f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:57.872170 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c257e2-456b-4b06-8b4f-dee3df2cc61f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:57.873631 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c257e2-456b-4b06-8b4f-dee3df2cc61f None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:57.875791 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 562/1109] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jun 16 17:31:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:57.881609 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6b5b5e-bcb0-4918-bdd4-1df5fc8fd8ed None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:57.967387 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:57.979742 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6b5b5e-bcb0-4918-bdd4-1df5fc8fd8ed None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:58.014017 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e6681f-b55c-4045-8d6d-8e329448cf7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:58.014701 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e6681f-b55c-4045-8d6d-8e329448cf7e None admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:58.015460 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e6681f-b55c-4045-8d6d-8e329448cf7e None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:58.015741 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e6681f-b55c-4045-8d6d-8e329448cf7e None admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:58.016740 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e6681f-b55c-4045-8d6d-8e329448cf7e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:58.018171 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e6681f-b55c-4045-8d6d-8e329448cf7e None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:58.037845 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 548/1110] 10.222.0.42 () {60 vars in 1132 bytes} [Thu Jun 16 17:31:57 2022] POST /identity/v3/users => generated 286 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:32:00.177947 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a297b4-90e8-413d-8a00-bf9279dc32f3 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:00.178454 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a297b4-90e8-413d-8a00-bf9279dc32f3 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:00.178569 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a297b4-90e8-413d-8a00-bf9279dc32f3 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:00.179878 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 563/1111] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:32:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:32:00.185152 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad586c2-0843-4dca-b1db-cfd2961041c2 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:00.185561 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad586c2-0843-4dca-b1db-cfd2961041c2 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:00.185713 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad586c2-0843-4dca-b1db-cfd2961041c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:00.203699 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-cad586c2-0843-4dca-b1db-cfd2961041c2 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:32:00.204915 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-cad586c2-0843-4dca-b1db-cfd2961041c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:32:00.270788 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 549/1112] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:32:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:32:00.294819 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a297b4-90e8-413d-8a00-bf9279dc32f3 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:32:00.313928 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:32:00.398088 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a297b4-90e8-413d-8a00-bf9279dc32f3 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:32:00.431161 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb6f346-43d8-47d1-a80c-99d0fde559b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:32:00.431932 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb6f346-43d8-47d1-a80c-99d0fde559b2 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:00.432216 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb6f346-43d8-47d1-a80c-99d0fde559b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:00.432520 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb6f346-43d8-47d1-a80c-99d0fde559b2 None admin] PATH_INFO: `/v3/roles/243fd0a11d94477986deb1dcfb3f31dd` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:00.433990 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb6f346-43d8-47d1-a80c-99d0fde559b2 None admin] RBAC: Authorizing `identity:get_role(role_id=243fd0a11d94477986deb1dcfb3f31dd)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:32:00.435356 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb6f346-43d8-47d1-a80c-99d0fde559b2 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:32:00.436894 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 564/1113] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jun 16 17:32:00 2022] GET /identity/v3/roles/243fd0a11d94477986deb1dcfb3f31dd => generated 223 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:32:00.442687 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad586c2-0843-4dca-b1db-cfd2961041c2 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:32:00.515498 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:32:00.540918 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad586c2-0843-4dca-b1db-cfd2961041c2 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:32:00.577406 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a760f8-47fa-4e61-805f-fd5fe28ed54e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:32:00.578227 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a760f8-47fa-4e61-805f-fd5fe28ed54e None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:00.578511 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a760f8-47fa-4e61-805f-fd5fe28ed54e None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:00.578784 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a760f8-47fa-4e61-805f-fd5fe28ed54e None admin] PATH_INFO: `/v3/users/42ed1fc898c149d4ad143bd7e4bb132e` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:00.591583 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a760f8-47fa-4e61-805f-fd5fe28ed54e None admin] RBAC: Authorizing `identity:get_user(user_id=42ed1fc898c149d4ad143bd7e4bb132e)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:32:00.593613 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a760f8-47fa-4e61-805f-fd5fe28ed54e None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:32:00.595254 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 550/1114] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jun 16 17:32:00 2022] GET /identity/v3/users/42ed1fc898c149d4ad143bd7e4bb132e => generated 286 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:32:00.601020 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb6f346-43d8-47d1-a80c-99d0fde559b2 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:32:00.683122 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:32:00.699431 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb6f346-43d8-47d1-a80c-99d0fde559b2 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:32:00.732985 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774576fb-bf97-478a-a704-dce6daad7ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:32:00.734662 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774576fb-bf97-478a-a704-dce6daad7ec7 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:00.734946 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774576fb-bf97-478a-a704-dce6daad7ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:00.735219 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774576fb-bf97-478a-a704-dce6daad7ec7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:00.802118 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774576fb-bf97-478a-a704-dce6daad7ec7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:32:00.803551 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774576fb-bf97-478a-a704-dce6daad7ec7 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:32:00.884205 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:32:00.901919 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 565/1115] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jun 16 17:32:00 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 17:32:00.908645 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a760f8-47fa-4e61-805f-fd5fe28ed54e None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:32:01.004553 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a760f8-47fa-4e61-805f-fd5fe28ed54e None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:32:01.032523 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:32:01.037931 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9c6d53-6dcf-4d32-b2a0-8236836f269b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:32:01.038775 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9c6d53-6dcf-4d32-b2a0-8236836f269b None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:01.039057 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9c6d53-6dcf-4d32-b2a0-8236836f269b None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:01.039331 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9c6d53-6dcf-4d32-b2a0-8236836f269b None admin] PATH_INFO: `/v3/projects/ac0783c70b014ada800a364d98afba49` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:01.040524 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9c6d53-6dcf-4d32-b2a0-8236836f269b None admin] RBAC: Authorizing `identity:get_project(project_id=ac0783c70b014ada800a364d98afba49)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:32:01.041953 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9c6d53-6dcf-4d32-b2a0-8236836f269b None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:32:01.043629 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 551/1116] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jun 16 17:32:00 2022] GET /identity/v3/projects/ac0783c70b014ada800a364d98afba49 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:32:01.050300 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774576fb-bf97-478a-a704-dce6daad7ec7 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:32:01.145830 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774576fb-bf97-478a-a704-dce6daad7ec7 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:32:01.179337 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62908dc-6628-4162-95ad-de585f890462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:32:01.179985 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62908dc-6628-4162-95ad-de585f890462 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:01.180269 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62908dc-6628-4162-95ad-de585f890462 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:01.180581 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62908dc-6628-4162-95ad-de585f890462 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:01.181401 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62908dc-6628-4162-95ad-de585f890462 None admin] RBAC: Adding query filter params (role.id=243fd0a11d94477986deb1dcfb3f31dd, scope.project.id=ac0783c70b014ada800a364d98afba49, user.id=42ed1fc898c149d4ad143bd7e4bb132e) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:32:01.181714 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62908dc-6628-4162-95ad-de585f890462 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:32:01.183147 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62908dc-6628-4162-95ad-de585f890462 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:32:01.189228 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 566/1117] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jun 16 17:32:01 2022] GET /identity/v3/role_assignments?user.id=42ed1fc898c149d4ad143bd7e4bb132e&scope.project.id=ac0783c70b014ada800a364d98afba49&role.id=243fd0a11d94477986deb1dcfb3f31dd => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:32:03.288386 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8512c9-3a7a-4e02-a58e-6e5d38349acf None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:03.288922 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8512c9-3a7a-4e02-a58e-6e5d38349acf None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:03.289042 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8512c9-3a7a-4e02-a58e-6e5d38349acf None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:03.290321 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 552/1118] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:32:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:32:03.295703 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d471533-b375-418a-8d19-4f601a82b5fd None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:03.296792 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d471533-b375-418a-8d19-4f601a82b5fd None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:03.296792 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d471533-b375-418a-8d19-4f601a82b5fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:03.314107 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-8d471533-b375-418a-8d19-4f601a82b5fd None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:32:03.315294 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-8d471533-b375-418a-8d19-4f601a82b5fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:32:03.359133 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:32:03.382718 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 567/1119] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:32:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:32:03.405842 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8512c9-3a7a-4e02-a58e-6e5d38349acf None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:32:03.503993 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8512c9-3a7a-4e02-a58e-6e5d38349acf None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:32:03.537258 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de3de8c-0300-4c9f-b69c-1862ae0ea933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:32:03.538099 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de3de8c-0300-4c9f-b69c-1862ae0ea933 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:03.538382 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de3de8c-0300-4c9f-b69c-1862ae0ea933 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:03.538707 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de3de8c-0300-4c9f-b69c-1862ae0ea933 None admin] PATH_INFO: `/v3/roles/243fd0a11d94477986deb1dcfb3f31dd` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:03.540109 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de3de8c-0300-4c9f-b69c-1862ae0ea933 None admin] RBAC: Authorizing `identity:get_role(role_id=243fd0a11d94477986deb1dcfb3f31dd)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:32:03.541505 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de3de8c-0300-4c9f-b69c-1862ae0ea933 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:32:03.542868 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 553/1120] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jun 16 17:32:03 2022] GET /identity/v3/roles/243fd0a11d94477986deb1dcfb3f31dd => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:32:03.549516 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d471533-b375-418a-8d19-4f601a82b5fd None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:32:03.646996 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d471533-b375-418a-8d19-4f601a82b5fd None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:32:03.681318 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5417f070-6ae9-4451-936a-5453922a95e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:32:03.682185 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5417f070-6ae9-4451-936a-5453922a95e2 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:03.682466 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5417f070-6ae9-4451-936a-5453922a95e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:03.682804 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5417f070-6ae9-4451-936a-5453922a95e2 None admin] PATH_INFO: `/v3/users/42ed1fc898c149d4ad143bd7e4bb132e` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:03.683913 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5417f070-6ae9-4451-936a-5453922a95e2 None admin] RBAC: Authorizing `identity:get_user(user_id=42ed1fc898c149d4ad143bd7e4bb132e)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:32:03.685361 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5417f070-6ae9-4451-936a-5453922a95e2 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:32:03.687506 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 568/1121] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jun 16 17:32:03 2022] GET /identity/v3/users/42ed1fc898c149d4ad143bd7e4bb132e => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:32:03.693339 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de3de8c-0300-4c9f-b69c-1862ae0ea933 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:32:03.761683 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:32:03.795339 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de3de8c-0300-4c9f-b69c-1862ae0ea933 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:32:03.829123 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cf9dba-3483-477b-8ae8-0d3ea412f420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:32:03.829965 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cf9dba-3483-477b-8ae8-0d3ea412f420 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:03.830249 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cf9dba-3483-477b-8ae8-0d3ea412f420 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:03.830525 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cf9dba-3483-477b-8ae8-0d3ea412f420 None admin] PATH_INFO: `/v3/projects/ac0783c70b014ada800a364d98afba49` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:03.831690 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cf9dba-3483-477b-8ae8-0d3ea412f420 None admin] RBAC: Authorizing `identity:get_project(project_id=ac0783c70b014ada800a364d98afba49)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:32:03.833133 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cf9dba-3483-477b-8ae8-0d3ea412f420 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:32:03.834810 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 554/1122] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jun 16 17:32:03 2022] GET /identity/v3/projects/ac0783c70b014ada800a364d98afba49 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:32:03.840743 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5417f070-6ae9-4451-936a-5453922a95e2 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:32:03.891124 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:32:03.945432 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5417f070-6ae9-4451-936a-5453922a95e2 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:32:03.977726 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6675b308-1cae-44b3-bf5f-d5f22b0c2fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:32:03.978606 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6675b308-1cae-44b3-bf5f-d5f22b0c2fa3 None admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:03.978892 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6675b308-1cae-44b3-bf5f-d5f22b0c2fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:03.979167 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6675b308-1cae-44b3-bf5f-d5f22b0c2fa3 None admin] PATH_INFO: `/v3/projects/ac0783c70b014ada800a364d98afba49/users/42ed1fc898c149d4ad143bd7e4bb132e/roles/243fd0a11d94477986deb1dcfb3f31dd` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:03.981392 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6675b308-1cae-44b3-bf5f-d5f22b0c2fa3 None admin] RBAC: Authorizing `identity:create_grant(project_id=ac0783c70b014ada800a364d98afba49, user_id=42ed1fc898c149d4ad143bd7e4bb132e, role_id=243fd0a11d94477986deb1dcfb3f31dd)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:32:03.982826 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6675b308-1cae-44b3-bf5f-d5f22b0c2fa3 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:32:03.995962 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 569/1123] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Jun 16 17:32:03 2022] PUT /identity/v3/projects/ac0783c70b014ada800a364d98afba49/users/42ed1fc898c149d4ad143bd7e4bb132e/roles/243fd0a11d94477986deb1dcfb3f31dd => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:32:06.117666 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5123a8-ca8c-49db-a769-8e47d69e85a9 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:06.118224 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5123a8-ca8c-49db-a769-8e47d69e85a9 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:06.118224 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5123a8-ca8c-49db-a769-8e47d69e85a9 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:06.119561 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 555/1124] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:32:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:32:06.125216 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ead518c-466a-4d96-8489-60343583fc2d None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:06.125635 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ead518c-466a-4d96-8489-60343583fc2d None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:06.125777 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ead518c-466a-4d96-8489-60343583fc2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:06.142978 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-0ead518c-466a-4d96-8489-60343583fc2d None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:32:06.144099 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-0ead518c-466a-4d96-8489-60343583fc2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:32:06.209816 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 570/1125] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:32:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:32:06.232667 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5123a8-ca8c-49db-a769-8e47d69e85a9 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:32:06.332945 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5123a8-ca8c-49db-a769-8e47d69e85a9 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:32:06.366842 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bb97a8-79d3-4617-ac04-1f396e0c9b89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:32:06.367683 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bb97a8-79d3-4617-ac04-1f396e0c9b89 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:06.367966 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bb97a8-79d3-4617-ac04-1f396e0c9b89 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:06.368237 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bb97a8-79d3-4617-ac04-1f396e0c9b89 None admin] PATH_INFO: `/v3/roles/243fd0a11d94477986deb1dcfb3f31dd` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:06.369739 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bb97a8-79d3-4617-ac04-1f396e0c9b89 None admin] RBAC: Authorizing `identity:get_role(role_id=243fd0a11d94477986deb1dcfb3f31dd)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:32:06.371088 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bb97a8-79d3-4617-ac04-1f396e0c9b89 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:32:06.372480 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 556/1126] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jun 16 17:32:06 2022] GET /identity/v3/roles/243fd0a11d94477986deb1dcfb3f31dd => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:32:06.378953 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ead518c-466a-4d96-8489-60343583fc2d None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:32:06.412392 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:32:06.476166 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ead518c-466a-4d96-8489-60343583fc2d None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:32:06.508983 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481e9262-d409-47dc-afef-4f29d0a269bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:32:06.509832 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481e9262-d409-47dc-afef-4f29d0a269bf None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:06.510113 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481e9262-d409-47dc-afef-4f29d0a269bf None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:06.510387 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481e9262-d409-47dc-afef-4f29d0a269bf None admin] PATH_INFO: `/v3/users/42ed1fc898c149d4ad143bd7e4bb132e` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:06.511551 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481e9262-d409-47dc-afef-4f29d0a269bf None admin] RBAC: Authorizing `identity:get_user(user_id=42ed1fc898c149d4ad143bd7e4bb132e)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:32:06.513009 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481e9262-d409-47dc-afef-4f29d0a269bf None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:32:06.515173 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 571/1127] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jun 16 17:32:06 2022] GET /identity/v3/users/42ed1fc898c149d4ad143bd7e4bb132e => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:32:06.521438 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bb97a8-79d3-4617-ac04-1f396e0c9b89 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:32:06.525145 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:32:06.619485 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bb97a8-79d3-4617-ac04-1f396e0c9b89 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:32:06.653751 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5748cc93-0b5f-4432-a242-5c6a89d24652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:32:06.654383 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5748cc93-0b5f-4432-a242-5c6a89d24652 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:06.654666 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5748cc93-0b5f-4432-a242-5c6a89d24652 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:06.654940 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5748cc93-0b5f-4432-a242-5c6a89d24652 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:06.722123 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5748cc93-0b5f-4432-a242-5c6a89d24652 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:32:06.723546 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5748cc93-0b5f-4432-a242-5c6a89d24652 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:32:06.726113 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:32:06.823706 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 557/1128] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jun 16 17:32:06 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 17:32:06.830251 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481e9262-d409-47dc-afef-4f29d0a269bf None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:32:06.934665 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481e9262-d409-47dc-afef-4f29d0a269bf None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:32:06.937806 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:32:06.968433 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8c661d-ca5e-410a-a964-6b7bfb3940ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:32:06.969360 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8c661d-ca5e-410a-a964-6b7bfb3940ec None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:06.969648 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8c661d-ca5e-410a-a964-6b7bfb3940ec None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:06.969930 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8c661d-ca5e-410a-a964-6b7bfb3940ec None admin] PATH_INFO: `/v3/projects/ac0783c70b014ada800a364d98afba49` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:06.971144 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8c661d-ca5e-410a-a964-6b7bfb3940ec None admin] RBAC: Authorizing `identity:get_project(project_id=ac0783c70b014ada800a364d98afba49)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:32:06.972659 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8c661d-ca5e-410a-a964-6b7bfb3940ec None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:32:06.974315 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 572/1129] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jun 16 17:32:06 2022] GET /identity/v3/projects/ac0783c70b014ada800a364d98afba49 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:32:06.980474 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5748cc93-0b5f-4432-a242-5c6a89d24652 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:32:07.077635 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5748cc93-0b5f-4432-a242-5c6a89d24652 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:32:07.079995 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:32:07.111748 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967b1073-cc31-4b87-b200-3418d70455c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:32:07.112411 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967b1073-cc31-4b87-b200-3418d70455c7 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:07.112698 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967b1073-cc31-4b87-b200-3418d70455c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:07.112972 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967b1073-cc31-4b87-b200-3418d70455c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:07.113796 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967b1073-cc31-4b87-b200-3418d70455c7 None admin] RBAC: Adding query filter params (role.id=243fd0a11d94477986deb1dcfb3f31dd, scope.project.id=ac0783c70b014ada800a364d98afba49, user.id=42ed1fc898c149d4ad143bd7e4bb132e) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:32:07.114109 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967b1073-cc31-4b87-b200-3418d70455c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:32:07.115535 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967b1073-cc31-4b87-b200-3418d70455c7 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:32:07.121285 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 558/1130] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jun 16 17:32:06 2022] GET /identity/v3/role_assignments?user.id=42ed1fc898c149d4ad143bd7e4bb132e&scope.project.id=ac0783c70b014ada800a364d98afba49&role.id=243fd0a11d94477986deb1dcfb3f31dd => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:32:09.251320 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21456bb-e61f-4d29-9820-edcbd304af53 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:09.251875 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21456bb-e61f-4d29-9820-edcbd304af53 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:09.251875 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21456bb-e61f-4d29-9820-edcbd304af53 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:09.253270 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 573/1131] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:32:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:32:09.258544 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489ccb35-8ce2-4e86-a61c-038b1b7f6bdc None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:09.258973 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489ccb35-8ce2-4e86-a61c-038b1b7f6bdc None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:09.259097 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489ccb35-8ce2-4e86-a61c-038b1b7f6bdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:09.276456 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-489ccb35-8ce2-4e86-a61c-038b1b7f6bdc None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:32:09.277641 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-489ccb35-8ce2-4e86-a61c-038b1b7f6bdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:32:09.341622 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 559/1132] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:32:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:32:09.366121 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21456bb-e61f-4d29-9820-edcbd304af53 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:32:09.471903 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21456bb-e61f-4d29-9820-edcbd304af53 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:32:09.517932 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab47b09-d0ba-4e6b-89b8-2200424e8557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:32:09.518649 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab47b09-d0ba-4e6b-89b8-2200424e8557 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:09.519199 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab47b09-d0ba-4e6b-89b8-2200424e8557 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:09.519494 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab47b09-d0ba-4e6b-89b8-2200424e8557 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:09.520751 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab47b09-d0ba-4e6b-89b8-2200424e8557 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:32:09.522385 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab47b09-d0ba-4e6b-89b8-2200424e8557 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:32:09.524000 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 574/1133] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jun 16 17:32:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:32:09.529816 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489ccb35-8ce2-4e86-a61c-038b1b7f6bdc None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:32:09.623758 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:32:09.628331 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489ccb35-8ce2-4e86-a61c-038b1b7f6bdc None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:32:09.661929 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408d6ad4-c81a-4512-99c6-034a70e9dbf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:32:09.662580 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408d6ad4-c81a-4512-99c6-034a70e9dbf1 None admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:09.662860 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408d6ad4-c81a-4512-99c6-034a70e9dbf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:09.663129 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408d6ad4-c81a-4512-99c6-034a70e9dbf1 None admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:09.663813 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408d6ad4-c81a-4512-99c6-034a70e9dbf1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:32:09.665251 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408d6ad4-c81a-4512-99c6-034a70e9dbf1 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:32:09.689709 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 560/1134] 10.222.0.42 () {60 vars in 1141 bytes} [Thu Jun 16 17:32:09 2022] POST /identity/v3/projects => generated 311 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:32:11.832588 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ec50bf-833c-4db6-93a1-b3289c9ad78a None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:11.833122 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ec50bf-833c-4db6-93a1-b3289c9ad78a None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:11.833122 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ec50bf-833c-4db6-93a1-b3289c9ad78a None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:11.834515 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 575/1135] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:32:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:32:11.839582 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1310420-9142-4b5f-913b-d43b2bf983cc None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:11.839984 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1310420-9142-4b5f-913b-d43b2bf983cc None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:11.840143 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1310420-9142-4b5f-913b-d43b2bf983cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:11.857068 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-e1310420-9142-4b5f-913b-d43b2bf983cc None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:32:11.858273 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-e1310420-9142-4b5f-913b-d43b2bf983cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:32:11.928273 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 561/1136] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:32:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:32:11.954095 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ec50bf-833c-4db6-93a1-b3289c9ad78a None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:32:11.996132 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:32:12.055083 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ec50bf-833c-4db6-93a1-b3289c9ad78a None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:32:12.092537 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1e139d-6bda-4801-97c6-49f305c6c840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:32:12.093347 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1e139d-6bda-4801-97c6-49f305c6c840 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:12.093634 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1e139d-6bda-4801-97c6-49f305c6c840 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:12.093916 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1e139d-6bda-4801-97c6-49f305c6c840 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:12.095087 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1e139d-6bda-4801-97c6-49f305c6c840 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:32:12.096544 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1e139d-6bda-4801-97c6-49f305c6c840 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:32:12.098160 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 576/1137] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jun 16 17:32:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:32:12.103904 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1310420-9142-4b5f-913b-d43b2bf983cc None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:32:12.181795 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:32:12.202037 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1310420-9142-4b5f-913b-d43b2bf983cc None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:32:12.234763 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3339b4-42d9-41eb-be80-7b3c9311f7e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:32:12.235420 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3339b4-42d9-41eb-be80-7b3c9311f7e8 None admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:12.235700 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3339b4-42d9-41eb-be80-7b3c9311f7e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:12.235973 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3339b4-42d9-41eb-be80-7b3c9311f7e8 None admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:12.236711 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3339b4-42d9-41eb-be80-7b3c9311f7e8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:32:12.238132 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3339b4-42d9-41eb-be80-7b3c9311f7e8 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:32:12.259773 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 562/1138] 10.222.0.42 () {60 vars in 1132 bytes} [Thu Jun 16 17:32:12 2022] POST /identity/v3/users => generated 286 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:32:14.444475 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a180dead-f1b9-479d-83aa-ecb2bfd33c56 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:14.445115 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a180dead-f1b9-479d-83aa-ecb2bfd33c56 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:14.445115 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a180dead-f1b9-479d-83aa-ecb2bfd33c56 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:14.446595 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 577/1139] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:32:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:32:14.452634 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1819f64b-492c-4068-80ee-19b9056a0dd7 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:14.453112 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1819f64b-492c-4068-80ee-19b9056a0dd7 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:14.453246 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1819f64b-492c-4068-80ee-19b9056a0dd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:14.470566 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-1819f64b-492c-4068-80ee-19b9056a0dd7 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:32:14.471752 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-1819f64b-492c-4068-80ee-19b9056a0dd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:32:14.537627 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 563/1140] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:32:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:32:14.560732 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a180dead-f1b9-479d-83aa-ecb2bfd33c56 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:32:14.659072 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a180dead-f1b9-479d-83aa-ecb2bfd33c56 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:32:14.667889 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:32:14.692780 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3ceca4-0134-45e6-b3e2-1727490bce91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:32:14.693558 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3ceca4-0134-45e6-b3e2-1727490bce91 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:14.693843 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3ceca4-0134-45e6-b3e2-1727490bce91 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:14.694119 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3ceca4-0134-45e6-b3e2-1727490bce91 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:14.704634 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3ceca4-0134-45e6-b3e2-1727490bce91 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:32:14.712116 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-7a3ceca4-0134-45e6-b3e2-1727490bce91 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:32:14.712116 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:32:14.712116 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:32:14.712116 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:32:14.712116 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:32:14.712116 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:32:14.712116 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:32:14.712116 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:32:14.712116 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:32:14.712116 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:32:14.712116 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:32:14.712116 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:32:14.712116 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:32:14.712116 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:32:14.712116 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:32:14.712116 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:32:14.713311 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:32:14.713311 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:32:14.713311 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:32:14.713311 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:32:14.713311 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:32:14.713311 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:32:14.713311 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:32:14.713311 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:32:14.713311 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:32:14.713311 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:32:14.713311 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:32:14.713311 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:32:14.713311 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:32:14.713311 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:32:14.713311 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:32:14.713311 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:32:14.714478 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:32:14.714478 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:32:14.714478 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:32:14.714478 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:32:14.714478 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:32:14.714478 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:32:14.714478 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:32:14.714478 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:32:14.714478 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:32:14.714478 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:32:14.714478 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:32:14.714478 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:32:14.714478 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 578/1141] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Jun 16 17:32:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:32:14.719876 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1819f64b-492c-4068-80ee-19b9056a0dd7 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:32:14.783731 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:32:14.820527 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1819f64b-492c-4068-80ee-19b9056a0dd7 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:32:14.853590 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aca7df7-4d86-4faf-b13e-e84a8eb4ab11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:32:14.854253 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aca7df7-4d86-4faf-b13e-e84a8eb4ab11 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:14.854538 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aca7df7-4d86-4faf-b13e-e84a8eb4ab11 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:14.854813 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aca7df7-4d86-4faf-b13e-e84a8eb4ab11 None admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:14.855557 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aca7df7-4d86-4faf-b13e-e84a8eb4ab11 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:32:14.855863 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aca7df7-4d86-4faf-b13e-e84a8eb4ab11 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:32:14.857259 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aca7df7-4d86-4faf-b13e-e84a8eb4ab11 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:32:14.865714 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 564/1142] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jun 16 17:32:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:32:14.871527 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3ceca4-0134-45e6-b3e2-1727490bce91 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:32:14.968879 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3ceca4-0134-45e6-b3e2-1727490bce91 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:32:15.004121 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7361ccb3-8d75-4c12-be4a-df65646b5ea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:32:15.005037 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7361ccb3-8d75-4c12-be4a-df65646b5ea7 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:15.005316 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7361ccb3-8d75-4c12-be4a-df65646b5ea7 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:15.005590 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7361ccb3-8d75-4c12-be4a-df65646b5ea7 None admin] PATH_INFO: `/v3/users/895d9bb4e18d4fb2aecefde5b5900b07` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:15.021648 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7361ccb3-8d75-4c12-be4a-df65646b5ea7 None admin] RBAC: Authorizing `identity:get_user(user_id=895d9bb4e18d4fb2aecefde5b5900b07)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:32:15.023089 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7361ccb3-8d75-4c12-be4a-df65646b5ea7 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:32:15.024790 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 579/1143] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jun 16 17:32:14 2022] GET /identity/v3/users/895d9bb4e18d4fb2aecefde5b5900b07 => generated 286 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:32:15.030668 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aca7df7-4d86-4faf-b13e-e84a8eb4ab11 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:32:15.127471 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aca7df7-4d86-4faf-b13e-e84a8eb4ab11 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:32:15.153320 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:32:15.161457 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26fa0e5-c624-4b8a-86cb-f266a67bd24e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:32:15.162091 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26fa0e5-c624-4b8a-86cb-f266a67bd24e None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:15.162372 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26fa0e5-c624-4b8a-86cb-f266a67bd24e None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:15.162645 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26fa0e5-c624-4b8a-86cb-f266a67bd24e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:15.228725 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26fa0e5-c624-4b8a-86cb-f266a67bd24e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:32:15.230111 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26fa0e5-c624-4b8a-86cb-f266a67bd24e None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:32:15.326714 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 565/1144] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jun 16 17:32:15 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 17:32:15.332716 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7361ccb3-8d75-4c12-be4a-df65646b5ea7 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:32:15.359611 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:32:15.429426 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7361ccb3-8d75-4c12-be4a-df65646b5ea7 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:32:15.461984 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2826c59d-6886-4481-be72-33de47c5fa15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:32:15.462821 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2826c59d-6886-4481-be72-33de47c5fa15 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:15.463100 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2826c59d-6886-4481-be72-33de47c5fa15 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:15.463374 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2826c59d-6886-4481-be72-33de47c5fa15 None admin] PATH_INFO: `/v3/projects/1efc51030fa1487182484b6d6cf444c5` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:15.464569 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2826c59d-6886-4481-be72-33de47c5fa15 None admin] RBAC: Authorizing `identity:get_project(project_id=1efc51030fa1487182484b6d6cf444c5)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:32:15.465990 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2826c59d-6886-4481-be72-33de47c5fa15 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:32:15.467707 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 580/1145] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jun 16 17:32:15 2022] GET /identity/v3/projects/1efc51030fa1487182484b6d6cf444c5 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:32:15.473742 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26fa0e5-c624-4b8a-86cb-f266a67bd24e None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:32:15.502324 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:32:15.573310 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26fa0e5-c624-4b8a-86cb-f266a67bd24e None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:32:15.606542 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0cc53b-d8fd-41d3-87b9-bbea0368d82a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:32:15.607187 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0cc53b-d8fd-41d3-87b9-bbea0368d82a None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:15.607473 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0cc53b-d8fd-41d3-87b9-bbea0368d82a None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:15.607750 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0cc53b-d8fd-41d3-87b9-bbea0368d82a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:15.609367 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0cc53b-d8fd-41d3-87b9-bbea0368d82a None admin] RBAC: Adding query filter params (role.id=3ca2dcbbd29d4a31a3ab78f2299bc239, scope.project.id=1efc51030fa1487182484b6d6cf444c5, user.id=895d9bb4e18d4fb2aecefde5b5900b07) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:32:15.609656 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0cc53b-d8fd-41d3-87b9-bbea0368d82a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:32:15.611092 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0cc53b-d8fd-41d3-87b9-bbea0368d82a None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:32:15.616509 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 566/1146] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jun 16 17:32:15 2022] GET /identity/v3/role_assignments?user.id=895d9bb4e18d4fb2aecefde5b5900b07&scope.project.id=1efc51030fa1487182484b6d6cf444c5&role.id=3ca2dcbbd29d4a31a3ab78f2299bc239 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:32:17.695974 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc4d916-3a18-42c0-9363-8f8b3a905718 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:17.696565 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc4d916-3a18-42c0-9363-8f8b3a905718 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:17.696565 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc4d916-3a18-42c0-9363-8f8b3a905718 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:17.697912 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 581/1147] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:32:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:32:17.703521 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e429684-9109-4a5c-af04-b66663c49944 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:17.703950 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e429684-9109-4a5c-af04-b66663c49944 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:17.704057 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e429684-9109-4a5c-af04-b66663c49944 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:17.721684 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-1e429684-9109-4a5c-af04-b66663c49944 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:32:17.722863 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-1e429684-9109-4a5c-af04-b66663c49944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:32:17.787982 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 567/1148] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:32:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:32:17.810449 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc4d916-3a18-42c0-9363-8f8b3a905718 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:32:17.905447 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:32:17.917600 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc4d916-3a18-42c0-9363-8f8b3a905718 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:32:17.951095 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51002894-4ba5-44eb-94a0-95f274f310db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:32:17.951808 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51002894-4ba5-44eb-94a0-95f274f310db None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:17.952701 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51002894-4ba5-44eb-94a0-95f274f310db None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:17.952981 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51002894-4ba5-44eb-94a0-95f274f310db None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:17.962739 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51002894-4ba5-44eb-94a0-95f274f310db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:32:17.970359 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-51002894-4ba5-44eb-94a0-95f274f310db None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:32:17.970359 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:32:17.970359 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:32:17.970359 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:32:17.970359 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:32:17.970359 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:32:17.970359 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:32:17.970359 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:32:17.970359 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:32:17.970359 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:32:17.970359 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:32:17.970359 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:32:17.970359 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:32:17.970359 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:32:17.970359 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:32:17.970359 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:32:17.971465 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:32:17.971465 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:32:17.971465 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:32:17.971465 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:32:17.971465 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:32:17.971465 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:32:17.971465 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:32:17.971465 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:32:17.971465 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:32:17.971465 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:32:17.971465 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:32:17.971465 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:32:17.971465 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:32:17.971465 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:32:17.971465 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:32:17.971465 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:32:17.972617 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:32:17.972617 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:32:17.972617 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:32:17.972617 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:32:17.972617 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:32:17.972617 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:32:17.972617 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:32:17.972617 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:32:17.972617 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:32:17.972617 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:32:17.972617 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:32:17.972617 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:32:17.972617 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 582/1149] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Jun 16 17:32:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:32:17.977738 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e429684-9109-4a5c-af04-b66663c49944 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:32:18.001372 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:32:18.075669 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e429684-9109-4a5c-af04-b66663c49944 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:32:18.109427 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea87635-0a69-40b1-b993-30b7e8e31864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:32:18.110089 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea87635-0a69-40b1-b993-30b7e8e31864 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:18.110374 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea87635-0a69-40b1-b993-30b7e8e31864 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:18.110669 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea87635-0a69-40b1-b993-30b7e8e31864 None admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:18.111398 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea87635-0a69-40b1-b993-30b7e8e31864 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:32:18.111706 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea87635-0a69-40b1-b993-30b7e8e31864 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:32:18.113096 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea87635-0a69-40b1-b993-30b7e8e31864 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:32:18.120948 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 568/1150] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jun 16 17:32:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:32:18.127408 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51002894-4ba5-44eb-94a0-95f274f310db None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:32:18.227703 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51002894-4ba5-44eb-94a0-95f274f310db None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:32:18.260838 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fa3621-1459-43e2-b1fe-c3a57bd44a96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:32:18.261697 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fa3621-1459-43e2-b1fe-c3a57bd44a96 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:18.261981 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fa3621-1459-43e2-b1fe-c3a57bd44a96 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:18.262260 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fa3621-1459-43e2-b1fe-c3a57bd44a96 None admin] PATH_INFO: `/v3/users/895d9bb4e18d4fb2aecefde5b5900b07` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:18.263432 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fa3621-1459-43e2-b1fe-c3a57bd44a96 None admin] RBAC: Authorizing `identity:get_user(user_id=895d9bb4e18d4fb2aecefde5b5900b07)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:32:18.264961 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fa3621-1459-43e2-b1fe-c3a57bd44a96 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:32:18.266574 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 583/1151] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jun 16 17:32:18 2022] GET /identity/v3/users/895d9bb4e18d4fb2aecefde5b5900b07 => generated 286 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:32:18.272662 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea87635-0a69-40b1-b993-30b7e8e31864 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:32:18.356560 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:32:18.373036 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea87635-0a69-40b1-b993-30b7e8e31864 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:32:18.406565 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef3955a-66a2-4cd2-8d9a-7dcf7014a4d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:32:18.407413 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef3955a-66a2-4cd2-8d9a-7dcf7014a4d9 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:18.407696 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef3955a-66a2-4cd2-8d9a-7dcf7014a4d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:18.407975 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef3955a-66a2-4cd2-8d9a-7dcf7014a4d9 None admin] PATH_INFO: `/v3/projects/1efc51030fa1487182484b6d6cf444c5` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:18.409175 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef3955a-66a2-4cd2-8d9a-7dcf7014a4d9 None admin] RBAC: Authorizing `identity:get_project(project_id=1efc51030fa1487182484b6d6cf444c5)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:32:18.410634 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef3955a-66a2-4cd2-8d9a-7dcf7014a4d9 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:32:18.412332 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 569/1152] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jun 16 17:32:18 2022] GET /identity/v3/projects/1efc51030fa1487182484b6d6cf444c5 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:32:18.418251 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fa3621-1459-43e2-b1fe-c3a57bd44a96 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:32:18.443507 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:32:18.516808 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fa3621-1459-43e2-b1fe-c3a57bd44a96 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:32:18.550698 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cc619c-3177-4b9a-ab60-68dac999d43c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:32:18.551576 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cc619c-3177-4b9a-ab60-68dac999d43c None admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:18.551862 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cc619c-3177-4b9a-ab60-68dac999d43c None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:18.552143 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cc619c-3177-4b9a-ab60-68dac999d43c None admin] PATH_INFO: `/v3/projects/1efc51030fa1487182484b6d6cf444c5/users/895d9bb4e18d4fb2aecefde5b5900b07/roles/3ca2dcbbd29d4a31a3ab78f2299bc239` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:18.554144 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cc619c-3177-4b9a-ab60-68dac999d43c None admin] RBAC: Authorizing `identity:create_grant(project_id=1efc51030fa1487182484b6d6cf444c5, user_id=895d9bb4e18d4fb2aecefde5b5900b07, role_id=3ca2dcbbd29d4a31a3ab78f2299bc239)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:32:18.555574 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cc619c-3177-4b9a-ab60-68dac999d43c None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:32:18.568275 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 584/1153] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Jun 16 17:32:18 2022] PUT /identity/v3/projects/1efc51030fa1487182484b6d6cf444c5/users/895d9bb4e18d4fb2aecefde5b5900b07/roles/3ca2dcbbd29d4a31a3ab78f2299bc239 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:32:20.732866 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316792bf-624d-4960-9b71-5b6465613b68 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:20.733582 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316792bf-624d-4960-9b71-5b6465613b68 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:20.733582 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316792bf-624d-4960-9b71-5b6465613b68 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:20.734849 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 570/1154] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:32:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:32:20.740661 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c7eced-aff1-4395-baf2-35163cf2e9cb None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:20.741096 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c7eced-aff1-4395-baf2-35163cf2e9cb None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:20.741226 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c7eced-aff1-4395-baf2-35163cf2e9cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:20.762518 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-96c7eced-aff1-4395-baf2-35163cf2e9cb None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:32:20.763706 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-96c7eced-aff1-4395-baf2-35163cf2e9cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:32:20.838859 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 585/1155] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:32:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:32:20.869744 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316792bf-624d-4960-9b71-5b6465613b68 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:32:20.975478 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316792bf-624d-4960-9b71-5b6465613b68 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:32:21.007635 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eec111c-708f-4760-86a9-76feae976533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:32:21.008348 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eec111c-708f-4760-86a9-76feae976533 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:21.008657 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eec111c-708f-4760-86a9-76feae976533 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:21.008928 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eec111c-708f-4760-86a9-76feae976533 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:21.019928 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eec111c-708f-4760-86a9-76feae976533 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:32:21.027371 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-6eec111c-708f-4760-86a9-76feae976533 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:32:21.027371 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:32:21.027371 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:32:21.027371 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:32:21.027371 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:32:21.027371 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:32:21.027371 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:32:21.027371 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:32:21.027371 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:32:21.027371 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:32:21.027371 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:32:21.027371 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:32:21.027371 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:32:21.027371 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:32:21.027371 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:32:21.027371 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:32:21.028601 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:32:21.028601 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:32:21.028601 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:32:21.028601 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:32:21.028601 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:32:21.028601 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:32:21.028601 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:32:21.028601 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:32:21.028601 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:32:21.028601 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:32:21.028601 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:32:21.028601 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:32:21.028601 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:32:21.028601 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:32:21.028601 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:32:21.028601 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:32:21.029488 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:32:21.029488 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:32:21.029488 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:32:21.029488 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:32:21.029488 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:32:21.029488 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:32:21.029488 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:32:21.029488 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:32:21.029488 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:32:21.029488 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:32:21.029488 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:32:21.029488 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:32:21.029488 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 571/1156] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Jun 16 17:32:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:32:21.034743 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c7eced-aff1-4395-baf2-35163cf2e9cb None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:32:21.048638 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:32:21.131929 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c7eced-aff1-4395-baf2-35163cf2e9cb None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:32:21.164963 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c31cba-9512-4cf0-a392-fa35c32baae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:32:21.165620 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c31cba-9512-4cf0-a392-fa35c32baae3 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:21.165899 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c31cba-9512-4cf0-a392-fa35c32baae3 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:21.166177 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c31cba-9512-4cf0-a392-fa35c32baae3 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:21.166908 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c31cba-9512-4cf0-a392-fa35c32baae3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:32:21.167215 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c31cba-9512-4cf0-a392-fa35c32baae3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:32:21.168601 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c31cba-9512-4cf0-a392-fa35c32baae3 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:32:21.176959 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 586/1157] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jun 16 17:32:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:32:21.182758 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eec111c-708f-4760-86a9-76feae976533 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:32:21.198138 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:32:21.300412 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eec111c-708f-4760-86a9-76feae976533 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:32:21.332803 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b616af70-d6be-4923-8f77-6a2b1de765b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:32:21.333655 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b616af70-d6be-4923-8f77-6a2b1de765b5 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:21.333934 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b616af70-d6be-4923-8f77-6a2b1de765b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:21.334196 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b616af70-d6be-4923-8f77-6a2b1de765b5 None admin] PATH_INFO: `/v3/users/895d9bb4e18d4fb2aecefde5b5900b07` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:21.335349 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b616af70-d6be-4923-8f77-6a2b1de765b5 None admin] RBAC: Authorizing `identity:get_user(user_id=895d9bb4e18d4fb2aecefde5b5900b07)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:32:21.336812 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b616af70-d6be-4923-8f77-6a2b1de765b5 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:32:21.338414 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 572/1158] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jun 16 17:32:21 2022] GET /identity/v3/users/895d9bb4e18d4fb2aecefde5b5900b07 => generated 286 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:32:21.344262 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c31cba-9512-4cf0-a392-fa35c32baae3 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:32:21.417806 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:32:21.443158 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c31cba-9512-4cf0-a392-fa35c32baae3 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:32:21.477104 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eff1b88-0e02-46ab-8603-4d00433d7172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:32:21.477747 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eff1b88-0e02-46ab-8603-4d00433d7172 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:21.478032 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eff1b88-0e02-46ab-8603-4d00433d7172 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:21.478311 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eff1b88-0e02-46ab-8603-4d00433d7172 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:21.544569 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eff1b88-0e02-46ab-8603-4d00433d7172 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:32:21.546690 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eff1b88-0e02-46ab-8603-4d00433d7172 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:32:21.617942 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:32:21.642913 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 587/1159] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jun 16 17:32:21 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 17:32:21.648849 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b616af70-d6be-4923-8f77-6a2b1de765b5 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:32:21.726674 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:32:21.745850 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b616af70-d6be-4923-8f77-6a2b1de765b5 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:32:21.778489 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc9761c-34c0-4f78-881d-d1ae63578ac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:32:21.779329 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc9761c-34c0-4f78-881d-d1ae63578ac4 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:21.779608 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc9761c-34c0-4f78-881d-d1ae63578ac4 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:21.779879 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc9761c-34c0-4f78-881d-d1ae63578ac4 None admin] PATH_INFO: `/v3/projects/1efc51030fa1487182484b6d6cf444c5` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:21.781680 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc9761c-34c0-4f78-881d-d1ae63578ac4 None admin] RBAC: Authorizing `identity:get_project(project_id=1efc51030fa1487182484b6d6cf444c5)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:32:21.783091 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc9761c-34c0-4f78-881d-d1ae63578ac4 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:32:21.784791 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 573/1160] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jun 16 17:32:21 2022] GET /identity/v3/projects/1efc51030fa1487182484b6d6cf444c5 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:32:21.791113 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eff1b88-0e02-46ab-8603-4d00433d7172 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:32:21.887987 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eff1b88-0e02-46ab-8603-4d00433d7172 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:32:21.924472 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc332e14-d44b-4cdb-bdc6-f4980a0a2d09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:32:21.925015 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc332e14-d44b-4cdb-bdc6-f4980a0a2d09 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:21.925302 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc332e14-d44b-4cdb-bdc6-f4980a0a2d09 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:21.925582 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc332e14-d44b-4cdb-bdc6-f4980a0a2d09 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:21.926404 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc332e14-d44b-4cdb-bdc6-f4980a0a2d09 None admin] RBAC: Adding query filter params (role.id=3ca2dcbbd29d4a31a3ab78f2299bc239, scope.project.id=1efc51030fa1487182484b6d6cf444c5, user.id=895d9bb4e18d4fb2aecefde5b5900b07) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:32:21.926720 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc332e14-d44b-4cdb-bdc6-f4980a0a2d09 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:32:21.928150 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc332e14-d44b-4cdb-bdc6-f4980a0a2d09 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:32:21.933974 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 588/1161] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jun 16 17:32:21 2022] GET /identity/v3/role_assignments?user.id=895d9bb4e18d4fb2aecefde5b5900b07&scope.project.id=1efc51030fa1487182484b6d6cf444c5&role.id=3ca2dcbbd29d4a31a3ab78f2299bc239 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:32:24.108100 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7e1731-b64b-4762-a7aa-d50bc6be0e78 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:24.108700 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7e1731-b64b-4762-a7aa-d50bc6be0e78 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:24.108799 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7e1731-b64b-4762-a7aa-d50bc6be0e78 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:24.110671 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 574/1162] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:32:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:32:24.115750 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1396efe5-da1a-411a-b9de-1cabe5b5ab03 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:24.116123 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1396efe5-da1a-411a-b9de-1cabe5b5ab03 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:24.116318 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1396efe5-da1a-411a-b9de-1cabe5b5ab03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:24.132934 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-1396efe5-da1a-411a-b9de-1cabe5b5ab03 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:32:24.134165 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-1396efe5-da1a-411a-b9de-1cabe5b5ab03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:32:24.163886 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:32:24.200108 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 589/1163] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:32:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:32:24.223265 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7e1731-b64b-4762-a7aa-d50bc6be0e78 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:32:24.322587 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7e1731-b64b-4762-a7aa-d50bc6be0e78 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:32:24.355966 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc20bb4a-31b9-4eb9-88b7-2484faa162ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:32:24.356634 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc20bb4a-31b9-4eb9-88b7-2484faa162ff None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:24.356916 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc20bb4a-31b9-4eb9-88b7-2484faa162ff None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:24.357192 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc20bb4a-31b9-4eb9-88b7-2484faa162ff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:24.374268 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:32:24.422483 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc20bb4a-31b9-4eb9-88b7-2484faa162ff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:32:24.423901 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc20bb4a-31b9-4eb9-88b7-2484faa162ff None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:32:24.521008 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 575/1164] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jun 16 17:32:24 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 17:32:24.526907 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1396efe5-da1a-411a-b9de-1cabe5b5ab03 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:32:24.623032 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1396efe5-da1a-411a-b9de-1cabe5b5ab03 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:32:24.657327 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2e822d-43c9-491e-a693-ace53e75bd45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:32:24.658118 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2e822d-43c9-491e-a693-ace53e75bd45 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:24.658405 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2e822d-43c9-491e-a693-ace53e75bd45 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:24.658682 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2e822d-43c9-491e-a693-ace53e75bd45 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:24.665193 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2e822d-43c9-491e-a693-ace53e75bd45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:32:24.666631 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2e822d-43c9-491e-a693-ace53e75bd45 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:32:24.680666 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-9f2e822d-43c9-491e-a693-ace53e75bd45 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 16 17:32:24.680666 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:32:24.680666 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jun 16 17:32:24.680666 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jun 16 17:32:24.680666 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:32:24.680666 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:32:24.680666 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:32:24.680666 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:32:24.680666 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jun 16 17:32:24.680666 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:32:24.680666 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:32:24.680666 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:32:24.680666 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:32:24.680666 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:32:24.680666 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:32:24.680666 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:32:24.680666 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:32:24.681970 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:32:24.681970 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:32:24.681970 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:32:24.681970 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:32:24.681970 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:32:24.681970 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:32:24.681970 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:32:24.681970 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:32:24.681970 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:32:24.681970 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:32:24.681970 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:32:24.681970 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:32:24.681970 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:32:24.681970 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:32:24.681970 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:32:24.681970 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:32:24.682926 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:32:24.682926 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:32:24.682926 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:32:24.682926 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:32:24.682926 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:32:24.682926 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:32:24.682926 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:32:24.682926 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:32:24.682926 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:32:24.682926 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:32:24.682926 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jun 16 17:32:24.682926 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:32:24.682926 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 16 17:32:24.682926 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:32:24.682926 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 590/1165] 10.222.0.42 () {56 vars in 1117 bytes} [Thu Jun 16 17:32:24 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:32:24.688343 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc20bb4a-31b9-4eb9-88b7-2484faa162ff None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:32:24.734709 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:32:24.783394 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc20bb4a-31b9-4eb9-88b7-2484faa162ff None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:32:24.816767 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbcac31-4b69-4f0d-93af-ac0046fbb415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:32:24.817406 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbcac31-4b69-4f0d-93af-ac0046fbb415 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:24.817688 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbcac31-4b69-4f0d-93af-ac0046fbb415 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:24.817960 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbcac31-4b69-4f0d-93af-ac0046fbb415 None admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:24.818671 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbcac31-4b69-4f0d-93af-ac0046fbb415 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:32:24.818970 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbcac31-4b69-4f0d-93af-ac0046fbb415 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:32:24.820309 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbcac31-4b69-4f0d-93af-ac0046fbb415 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:32:24.827888 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 576/1166] 10.222.0.42 () {56 vars in 1115 bytes} [Thu Jun 16 17:32:24 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:32:27.034241 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24726f0f-e471-4c45-9551-4efe4197ced9 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:27.034785 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24726f0f-e471-4c45-9551-4efe4197ced9 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:27.034785 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24726f0f-e471-4c45-9551-4efe4197ced9 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:27.036157 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 591/1167] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:32:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:32:27.041269 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c78883e-0bb5-4bed-bc79-40627086a496 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:27.041660 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c78883e-0bb5-4bed-bc79-40627086a496 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:27.041831 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c78883e-0bb5-4bed-bc79-40627086a496 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:27.058175 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-5c78883e-0bb5-4bed-bc79-40627086a496 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:32:27.059344 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-5c78883e-0bb5-4bed-bc79-40627086a496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:32:27.123348 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 577/1168] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:32:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:32:27.148337 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24726f0f-e471-4c45-9551-4efe4197ced9 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:32:27.177065 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:32:27.249805 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24726f0f-e471-4c45-9551-4efe4197ced9 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:32:27.283004 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c49745-a07b-44e7-aeac-ff91de83872d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:32:27.283665 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c49745-a07b-44e7-aeac-ff91de83872d None admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:27.283944 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c49745-a07b-44e7-aeac-ff91de83872d None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:27.284219 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c49745-a07b-44e7-aeac-ff91de83872d None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:27.284987 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c49745-a07b-44e7-aeac-ff91de83872d None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:32:27.286362 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c49745-a07b-44e7-aeac-ff91de83872d None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:32:27.302092 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 592/1169] 10.222.0.42 () {60 vars in 1138 bytes} [Thu Jun 16 17:32:27 2022] POST /identity/v3/domains => generated 267 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:32:29.504317 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492ea77f-f418-45df-b0e6-2a66982eb844 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:29.504917 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492ea77f-f418-45df-b0e6-2a66982eb844 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:29.504917 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492ea77f-f418-45df-b0e6-2a66982eb844 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:29.506276 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 578/1170] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:32:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:32:29.511337 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83a89d2-9faf-40df-8f1a-b338f60642ce None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:29.511714 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83a89d2-9faf-40df-8f1a-b338f60642ce None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:29.511892 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83a89d2-9faf-40df-8f1a-b338f60642ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:29.529411 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-e83a89d2-9faf-40df-8f1a-b338f60642ce None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:32:29.531292 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-e83a89d2-9faf-40df-8f1a-b338f60642ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:32:29.596109 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 593/1171] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:32:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:32:29.619349 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492ea77f-f418-45df-b0e6-2a66982eb844 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:32:29.663274 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:32:29.718967 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492ea77f-f418-45df-b0e6-2a66982eb844 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:32:29.751804 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d83a280-252b-4348-8b38-28e46ce1a186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:32:29.752664 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d83a280-252b-4348-8b38-28e46ce1a186 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:29.752943 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d83a280-252b-4348-8b38-28e46ce1a186 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:29.753210 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d83a280-252b-4348-8b38-28e46ce1a186 None admin] PATH_INFO: `/v3/domains/2808439752ab4b938b25de6d2a1f509f` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:29.763202 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d83a280-252b-4348-8b38-28e46ce1a186 None admin] RBAC: Authorizing `identity:get_domain(domain_id=2808439752ab4b938b25de6d2a1f509f)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:32:29.765405 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d83a280-252b-4348-8b38-28e46ce1a186 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:32:29.766849 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 579/1172] 10.222.0.42 () {56 vars in 1183 bytes} [Thu Jun 16 17:32:29 2022] GET /identity/v3/domains/2808439752ab4b938b25de6d2a1f509f => generated 267 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:32:29.772841 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83a89d2-9faf-40df-8f1a-b338f60642ce None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:32:29.791135 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:32:29.877014 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83a89d2-9faf-40df-8f1a-b338f60642ce None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:32:29.910999 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba6ad60-432a-4b0e-ac94-1c2df3272c84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:32:29.911646 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba6ad60-432a-4b0e-ac94-1c2df3272c84 None admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:29.911931 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba6ad60-432a-4b0e-ac94-1c2df3272c84 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:29.912212 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba6ad60-432a-4b0e-ac94-1c2df3272c84 None admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:29.912945 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba6ad60-432a-4b0e-ac94-1c2df3272c84 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:32:29.914367 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba6ad60-432a-4b0e-ac94-1c2df3272c84 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:32:29.939058 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 594/1173] 10.222.0.42 () {60 vars in 1141 bytes} [Thu Jun 16 17:32:29 2022] POST /identity/v3/projects => generated 361 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:32:32.134374 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148d83ce-3eae-443e-b2ed-aea2a2e1fde9 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:32.134876 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148d83ce-3eae-443e-b2ed-aea2a2e1fde9 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:32.134876 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148d83ce-3eae-443e-b2ed-aea2a2e1fde9 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:32.136205 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 580/1174] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:32:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:32:32.142710 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8e21fd-537c-48d1-947a-cf8c11a92e1d None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:32.143143 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8e21fd-537c-48d1-947a-cf8c11a92e1d None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:32.143266 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8e21fd-537c-48d1-947a-cf8c11a92e1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:32.159781 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-3a8e21fd-537c-48d1-947a-cf8c11a92e1d None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:32:32.161007 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-3a8e21fd-537c-48d1-947a-cf8c11a92e1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:32:32.221923 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:32:32.227289 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 595/1175] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:32:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:32:32.252279 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148d83ce-3eae-443e-b2ed-aea2a2e1fde9 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:32:32.352164 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148d83ce-3eae-443e-b2ed-aea2a2e1fde9 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:32:32.353690 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:32:32.386399 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ed7125-6343-44f6-869e-fbbb225bd4fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:32:32.387180 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ed7125-6343-44f6-869e-fbbb225bd4fb None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:32.387464 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ed7125-6343-44f6-869e-fbbb225bd4fb None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:32.387743 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ed7125-6343-44f6-869e-fbbb225bd4fb None admin] PATH_INFO: `/v3/domains/2808439752ab4b938b25de6d2a1f509f` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:32.388934 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ed7125-6343-44f6-869e-fbbb225bd4fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=2808439752ab4b938b25de6d2a1f509f)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:32:32.391000 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ed7125-6343-44f6-869e-fbbb225bd4fb None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:32:32.392888 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 581/1176] 10.222.0.42 () {56 vars in 1183 bytes} [Thu Jun 16 17:32:32 2022] GET /identity/v3/domains/2808439752ab4b938b25de6d2a1f509f => generated 267 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:32:32.398788 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8e21fd-537c-48d1-947a-cf8c11a92e1d None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:32:32.495162 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8e21fd-537c-48d1-947a-cf8c11a92e1d None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:32:32.528590 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc85481-3a48-413c-9d5e-7ed65746e6ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:32:32.529248 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc85481-3a48-413c-9d5e-7ed65746e6ed None admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:32.529533 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc85481-3a48-413c-9d5e-7ed65746e6ed None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:32.529813 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc85481-3a48-413c-9d5e-7ed65746e6ed None admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:32.530536 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc85481-3a48-413c-9d5e-7ed65746e6ed None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:32:32.531959 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc85481-3a48-413c-9d5e-7ed65746e6ed None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:32:32.551464 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 596/1177] 10.222.0.42 () {60 vars in 1132 bytes} [Thu Jun 16 17:32:32 2022] POST /identity/v3/users => generated 311 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:32:34.739620 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3291e48-7e74-48e3-8a16-cc9ddc3083b5 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:34.740192 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3291e48-7e74-48e3-8a16-cc9ddc3083b5 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:34.740192 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3291e48-7e74-48e3-8a16-cc9ddc3083b5 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:34.741578 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 582/1178] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:32:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:32:34.746677 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968877b3-20d2-464d-aeaa-d98e74eba286 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:34.747050 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968877b3-20d2-464d-aeaa-d98e74eba286 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:34.747243 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968877b3-20d2-464d-aeaa-d98e74eba286 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:34.763736 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-968877b3-20d2-464d-aeaa-d98e74eba286 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:32:34.765574 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-968877b3-20d2-464d-aeaa-d98e74eba286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:32:34.814693 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:32:34.832181 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 597/1179] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:32:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:32:34.855981 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3291e48-7e74-48e3-8a16-cc9ddc3083b5 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:32:34.956592 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3291e48-7e74-48e3-8a16-cc9ddc3083b5 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:32:34.992436 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfdf214-2d1b-4399-9116-832269f139c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:32:34.993158 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfdf214-2d1b-4399-9116-832269f139c6 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:34.993447 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfdf214-2d1b-4399-9116-832269f139c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:34.993726 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfdf214-2d1b-4399-9116-832269f139c6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:35.004725 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfdf214-2d1b-4399-9116-832269f139c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:32:35.012117 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-fdfdf214-2d1b-4399-9116-832269f139c6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:32:35.012117 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:32:35.012117 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:32:35.012117 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:32:35.012117 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:32:35.012117 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:32:35.012117 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:32:35.012117 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:32:35.012117 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:32:35.012117 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:32:35.012117 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:32:35.012117 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:32:35.012117 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:32:35.012117 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:32:35.012117 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:32:35.012117 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:32:35.013233 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:32:35.013233 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:32:35.013233 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:32:35.013233 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:32:35.013233 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:32:35.013233 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:32:35.013233 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:32:35.013233 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:32:35.013233 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:32:35.013233 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:32:35.013233 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:32:35.013233 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:32:35.013233 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:32:35.013233 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:32:35.013233 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:32:35.013233 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:32:35.014152 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:32:35.014152 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:32:35.014152 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:32:35.014152 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:32:35.014152 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:32:35.014152 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:32:35.014152 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:32:35.014152 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:32:35.014152 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:32:35.014152 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:32:35.014152 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:32:35.014152 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:32:35.014152 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 583/1180] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Jun 16 17:32:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:32:35.019578 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968877b3-20d2-464d-aeaa-d98e74eba286 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:32:35.115270 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968877b3-20d2-464d-aeaa-d98e74eba286 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:32:35.147316 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2eb897-4362-452c-8e9a-b642146fa2bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:32:35.147970 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2eb897-4362-452c-8e9a-b642146fa2bc None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:35.148249 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2eb897-4362-452c-8e9a-b642146fa2bc None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:35.148554 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2eb897-4362-452c-8e9a-b642146fa2bc None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:35.149282 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2eb897-4362-452c-8e9a-b642146fa2bc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:32:35.149588 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2eb897-4362-452c-8e9a-b642146fa2bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:32:35.150938 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2eb897-4362-452c-8e9a-b642146fa2bc None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:32:35.159377 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 598/1181] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jun 16 17:32:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:32:35.165256 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfdf214-2d1b-4399-9116-832269f139c6 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:32:35.195171 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:32:35.262538 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfdf214-2d1b-4399-9116-832269f139c6 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:32:35.295481 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068db92e-ea1b-4497-a73d-09d58f921557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:32:35.296902 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068db92e-ea1b-4497-a73d-09d58f921557 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:35.297187 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068db92e-ea1b-4497-a73d-09d58f921557 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:35.297454 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068db92e-ea1b-4497-a73d-09d58f921557 None admin] PATH_INFO: `/v3/users/f056650f6e1049cab19652c6818ee974` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:35.310874 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068db92e-ea1b-4497-a73d-09d58f921557 None admin] RBAC: Authorizing `identity:get_user(user_id=f056650f6e1049cab19652c6818ee974)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:32:35.312622 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068db92e-ea1b-4497-a73d-09d58f921557 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:32:35.314257 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 584/1182] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jun 16 17:32:35 2022] GET /identity/v3/users/f056650f6e1049cab19652c6818ee974 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:32:35.320294 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2eb897-4362-452c-8e9a-b642146fa2bc None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:32:35.368560 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:32:35.418235 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2eb897-4362-452c-8e9a-b642146fa2bc None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:32:35.449603 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1047db4d-4223-4dba-b734-1b32958adad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:32:35.450239 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1047db4d-4223-4dba-b734-1b32958adad7 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:35.450518 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1047db4d-4223-4dba-b734-1b32958adad7 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:35.450790 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1047db4d-4223-4dba-b734-1b32958adad7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:35.515768 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1047db4d-4223-4dba-b734-1b32958adad7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:32:35.517157 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1047db4d-4223-4dba-b734-1b32958adad7 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:32:35.561852 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:32:35.612214 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 599/1183] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jun 16 17:32:35 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 17:32:35.618127 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068db92e-ea1b-4497-a73d-09d58f921557 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:32:35.704346 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:32:35.716264 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068db92e-ea1b-4497-a73d-09d58f921557 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:32:35.750149 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837aa0ca-2d91-4353-9b2c-7394f1354711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:32:35.750993 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837aa0ca-2d91-4353-9b2c-7394f1354711 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:35.751278 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837aa0ca-2d91-4353-9b2c-7394f1354711 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:35.751555 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837aa0ca-2d91-4353-9b2c-7394f1354711 None admin] PATH_INFO: `/v3/projects/b0db41fa8dcf4888b56f52ba630c67bf` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:35.752752 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837aa0ca-2d91-4353-9b2c-7394f1354711 None admin] RBAC: Authorizing `identity:get_project(project_id=b0db41fa8dcf4888b56f52ba630c67bf)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:32:35.754175 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837aa0ca-2d91-4353-9b2c-7394f1354711 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:32:35.755869 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 585/1184] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jun 16 17:32:35 2022] GET /identity/v3/projects/b0db41fa8dcf4888b56f52ba630c67bf => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:32:35.761949 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1047db4d-4223-4dba-b734-1b32958adad7 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:32:35.860384 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1047db4d-4223-4dba-b734-1b32958adad7 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:32:35.894946 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777cc785-9c15-4280-a490-eee1df2bff96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:32:35.895599 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777cc785-9c15-4280-a490-eee1df2bff96 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:35.895882 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777cc785-9c15-4280-a490-eee1df2bff96 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:35.896159 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777cc785-9c15-4280-a490-eee1df2bff96 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:35.897049 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777cc785-9c15-4280-a490-eee1df2bff96 None admin] RBAC: Adding query filter params (role.id=3ca2dcbbd29d4a31a3ab78f2299bc239, scope.project.id=b0db41fa8dcf4888b56f52ba630c67bf, user.id=f056650f6e1049cab19652c6818ee974) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:32:35.897365 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777cc785-9c15-4280-a490-eee1df2bff96 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:32:35.898785 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777cc785-9c15-4280-a490-eee1df2bff96 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:32:35.904092 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 600/1185] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jun 16 17:32:35 2022] GET /identity/v3/role_assignments?user.id=f056650f6e1049cab19652c6818ee974&scope.project.id=b0db41fa8dcf4888b56f52ba630c67bf&role.id=3ca2dcbbd29d4a31a3ab78f2299bc239 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:32:38.004036 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d8581b-7388-494e-b722-b9b17d67310c None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:38.004610 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d8581b-7388-494e-b722-b9b17d67310c None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:38.004610 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d8581b-7388-494e-b722-b9b17d67310c None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:38.005982 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 586/1186] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:32:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:32:38.011290 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aaf06a-03c4-4165-bf14-12d622306252 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:38.011721 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aaf06a-03c4-4165-bf14-12d622306252 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:38.011843 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aaf06a-03c4-4165-bf14-12d622306252 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:38.029553 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-a8aaf06a-03c4-4165-bf14-12d622306252 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:32:38.030862 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-a8aaf06a-03c4-4165-bf14-12d622306252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:32:38.038025 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:32:38.096614 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 601/1187] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:32:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:32:38.119345 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d8581b-7388-494e-b722-b9b17d67310c None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:32:38.219546 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d8581b-7388-494e-b722-b9b17d67310c None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:32:38.252572 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc125f0-a97e-4a6c-a955-5a29c1817c10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:32:38.253346 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc125f0-a97e-4a6c-a955-5a29c1817c10 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:38.253625 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc125f0-a97e-4a6c-a955-5a29c1817c10 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:38.253895 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc125f0-a97e-4a6c-a955-5a29c1817c10 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:38.263644 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc125f0-a97e-4a6c-a955-5a29c1817c10 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:32:38.272020 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-6fc125f0-a97e-4a6c-a955-5a29c1817c10 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:32:38.272020 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:32:38.272020 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:32:38.272020 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:32:38.272020 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:32:38.272020 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:32:38.272020 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:32:38.272020 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:32:38.272020 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:32:38.272020 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:32:38.272020 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:32:38.272020 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:32:38.272020 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:32:38.272020 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:32:38.272020 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:32:38.272020 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:32:38.273130 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:32:38.273130 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:32:38.273130 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:32:38.273130 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:32:38.273130 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:32:38.273130 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:32:38.273130 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:32:38.273130 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:32:38.273130 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:32:38.273130 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:32:38.273130 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:32:38.273130 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:32:38.273130 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:32:38.273130 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:32:38.273130 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:32:38.273130 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:32:38.273987 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:32:38.273987 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:32:38.273987 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:32:38.273987 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:32:38.273987 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:32:38.273987 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:32:38.273987 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:32:38.273987 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:32:38.273987 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:32:38.273987 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:32:38.273987 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:32:38.273987 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:32:38.273987 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 587/1188] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Jun 16 17:32:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:32:38.279540 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8aaf06a-03c4-4165-bf14-12d622306252 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:32:38.376038 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8aaf06a-03c4-4165-bf14-12d622306252 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:32:38.409166 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db6f06f-4fee-43f7-ad8d-a4a6ea27b8e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:32:38.409815 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db6f06f-4fee-43f7-ad8d-a4a6ea27b8e0 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:38.410094 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db6f06f-4fee-43f7-ad8d-a4a6ea27b8e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:38.410366 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db6f06f-4fee-43f7-ad8d-a4a6ea27b8e0 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:38.411092 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db6f06f-4fee-43f7-ad8d-a4a6ea27b8e0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:32:38.411396 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db6f06f-4fee-43f7-ad8d-a4a6ea27b8e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:32:38.412784 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db6f06f-4fee-43f7-ad8d-a4a6ea27b8e0 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:32:38.421121 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 602/1189] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jun 16 17:32:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:32:38.426741 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc125f0-a97e-4a6c-a955-5a29c1817c10 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:32:38.445567 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:32:38.529084 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc125f0-a97e-4a6c-a955-5a29c1817c10 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:32:38.561936 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f64f7c-83db-4f10-8f5c-abff1c219e95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:32:38.562858 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f64f7c-83db-4f10-8f5c-abff1c219e95 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:38.563135 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f64f7c-83db-4f10-8f5c-abff1c219e95 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:38.563402 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f64f7c-83db-4f10-8f5c-abff1c219e95 None admin] PATH_INFO: `/v3/users/f056650f6e1049cab19652c6818ee974` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:38.564601 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f64f7c-83db-4f10-8f5c-abff1c219e95 None admin] RBAC: Authorizing `identity:get_user(user_id=f056650f6e1049cab19652c6818ee974)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:32:38.566022 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f64f7c-83db-4f10-8f5c-abff1c219e95 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:32:38.567625 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 588/1190] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jun 16 17:32:38 2022] GET /identity/v3/users/f056650f6e1049cab19652c6818ee974 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:32:38.573387 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db6f06f-4fee-43f7-ad8d-a4a6ea27b8e0 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:32:38.577267 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:32:38.671207 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db6f06f-4fee-43f7-ad8d-a4a6ea27b8e0 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:32:38.703655 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11a4af9-6fb7-4494-b4a5-21a824c24e13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:32:38.704527 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11a4af9-6fb7-4494-b4a5-21a824c24e13 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:38.704813 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11a4af9-6fb7-4494-b4a5-21a824c24e13 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:38.705089 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11a4af9-6fb7-4494-b4a5-21a824c24e13 None admin] PATH_INFO: `/v3/projects/b0db41fa8dcf4888b56f52ba630c67bf` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:38.706288 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11a4af9-6fb7-4494-b4a5-21a824c24e13 None admin] RBAC: Authorizing `identity:get_project(project_id=b0db41fa8dcf4888b56f52ba630c67bf)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:32:38.707711 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11a4af9-6fb7-4494-b4a5-21a824c24e13 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:32:38.709418 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 603/1191] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jun 16 17:32:38 2022] GET /identity/v3/projects/b0db41fa8dcf4888b56f52ba630c67bf => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:32:38.715235 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f64f7c-83db-4f10-8f5c-abff1c219e95 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:32:38.797048 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:32:38.813766 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f64f7c-83db-4f10-8f5c-abff1c219e95 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:32:38.846230 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5dcd6a-c52b-4b2b-bb05-ed9a704a3ed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:32:38.847105 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5dcd6a-c52b-4b2b-bb05-ed9a704a3ed7 None admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:38.847398 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5dcd6a-c52b-4b2b-bb05-ed9a704a3ed7 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:38.847675 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5dcd6a-c52b-4b2b-bb05-ed9a704a3ed7 None admin] PATH_INFO: `/v3/projects/b0db41fa8dcf4888b56f52ba630c67bf/users/f056650f6e1049cab19652c6818ee974/roles/3ca2dcbbd29d4a31a3ab78f2299bc239` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:38.849639 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5dcd6a-c52b-4b2b-bb05-ed9a704a3ed7 None admin] RBAC: Authorizing `identity:create_grant(project_id=b0db41fa8dcf4888b56f52ba630c67bf, user_id=f056650f6e1049cab19652c6818ee974, role_id=3ca2dcbbd29d4a31a3ab78f2299bc239)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:32:38.851102 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5dcd6a-c52b-4b2b-bb05-ed9a704a3ed7 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:32:38.863570 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 589/1192] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Jun 16 17:32:38 2022] PUT /identity/v3/projects/b0db41fa8dcf4888b56f52ba630c67bf/users/f056650f6e1049cab19652c6818ee974/roles/3ca2dcbbd29d4a31a3ab78f2299bc239 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:32:40.959487 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756f1b07-21a6-40ca-83db-1ef476a22e78 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:40.960029 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756f1b07-21a6-40ca-83db-1ef476a22e78 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:40.960029 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756f1b07-21a6-40ca-83db-1ef476a22e78 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:40.961438 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 604/1193] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:32:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:32:40.966462 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6f1714-9335-476a-9f57-b934dc4ca58e None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:40.966836 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6f1714-9335-476a-9f57-b934dc4ca58e None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:40.967018 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6f1714-9335-476a-9f57-b934dc4ca58e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:40.983864 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-9c6f1714-9335-476a-9f57-b934dc4ca58e None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:32:40.985063 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-9c6f1714-9335-476a-9f57-b934dc4ca58e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:32:41.050022 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 590/1194] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:32:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:32:41.074262 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756f1b07-21a6-40ca-83db-1ef476a22e78 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:32:41.146718 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:32:41.180162 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756f1b07-21a6-40ca-83db-1ef476a22e78 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:32:41.227275 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d639f3ab-ba4d-42a5-a707-7fcc57579a05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:32:41.228489 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d639f3ab-ba4d-42a5-a707-7fcc57579a05 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:41.228972 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d639f3ab-ba4d-42a5-a707-7fcc57579a05 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:41.229254 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d639f3ab-ba4d-42a5-a707-7fcc57579a05 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:41.243367 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d639f3ab-ba4d-42a5-a707-7fcc57579a05 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:32:41.253262 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-d639f3ab-ba4d-42a5-a707-7fcc57579a05 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:32:41.253262 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:32:41.253262 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:32:41.253262 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:32:41.253262 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:32:41.253262 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:32:41.253262 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:32:41.253262 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:32:41.253262 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:32:41.253262 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:32:41.253262 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:32:41.253262 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:32:41.253262 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:32:41.253262 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:32:41.253262 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:32:41.253262 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:32:41.254409 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:32:41.254409 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:32:41.254409 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:32:41.254409 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:32:41.254409 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:32:41.254409 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:32:41.254409 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:32:41.254409 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:32:41.254409 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:32:41.254409 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:32:41.254409 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:32:41.254409 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:32:41.254409 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:32:41.254409 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:32:41.254409 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:32:41.254409 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:32:41.255537 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:32:41.255537 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:32:41.255537 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:32:41.255537 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:32:41.255537 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:32:41.255537 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:32:41.255537 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:32:41.255537 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:32:41.255537 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:32:41.255537 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:32:41.255537 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:32:41.255537 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:32:41.255537 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 605/1195] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Jun 16 17:32:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:32:41.261200 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6f1714-9335-476a-9f57-b934dc4ca58e None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:32:41.338940 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:32:41.364892 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6f1714-9335-476a-9f57-b934dc4ca58e None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:32:41.398989 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd8a094-d227-4742-9c7f-7aaa6713cf13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:32:41.399585 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd8a094-d227-4742-9c7f-7aaa6713cf13 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:41.399864 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd8a094-d227-4742-9c7f-7aaa6713cf13 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:41.400136 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd8a094-d227-4742-9c7f-7aaa6713cf13 None admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:41.400901 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd8a094-d227-4742-9c7f-7aaa6713cf13 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:32:41.401202 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd8a094-d227-4742-9c7f-7aaa6713cf13 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:32:41.402552 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd8a094-d227-4742-9c7f-7aaa6713cf13 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:32:41.410497 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 591/1196] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jun 16 17:32:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:32:41.416353 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d639f3ab-ba4d-42a5-a707-7fcc57579a05 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:32:41.513218 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d639f3ab-ba4d-42a5-a707-7fcc57579a05 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:32:41.541304 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:32:41.547074 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d8d61e-a7a3-48ed-ad0c-eb7fcef6fbf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:32:41.547935 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d8d61e-a7a3-48ed-ad0c-eb7fcef6fbf9 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:41.548216 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d8d61e-a7a3-48ed-ad0c-eb7fcef6fbf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:41.548522 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d8d61e-a7a3-48ed-ad0c-eb7fcef6fbf9 None admin] PATH_INFO: `/v3/users/f056650f6e1049cab19652c6818ee974` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:41.549695 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d8d61e-a7a3-48ed-ad0c-eb7fcef6fbf9 None admin] RBAC: Authorizing `identity:get_user(user_id=f056650f6e1049cab19652c6818ee974)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:32:41.551155 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d8d61e-a7a3-48ed-ad0c-eb7fcef6fbf9 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:32:41.552807 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 606/1197] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jun 16 17:32:41 2022] GET /identity/v3/users/f056650f6e1049cab19652c6818ee974 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:32:41.558564 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd8a094-d227-4742-9c7f-7aaa6713cf13 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:32:41.668380 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd8a094-d227-4742-9c7f-7aaa6713cf13 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:32:41.701259 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f49f61-8845-478b-9d8e-c7e099feaca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:32:41.702000 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f49f61-8845-478b-9d8e-c7e099feaca4 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:41.702285 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f49f61-8845-478b-9d8e-c7e099feaca4 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:41.702557 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f49f61-8845-478b-9d8e-c7e099feaca4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:41.705880 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:32:41.769272 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f49f61-8845-478b-9d8e-c7e099feaca4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:32:41.770709 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f49f61-8845-478b-9d8e-c7e099feaca4 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:32:41.867889 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 592/1198] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jun 16 17:32:41 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 17:32:41.873690 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d8d61e-a7a3-48ed-ad0c-eb7fcef6fbf9 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:32:41.973978 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d8d61e-a7a3-48ed-ad0c-eb7fcef6fbf9 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:32:42.006253 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c36053-0383-4302-8753-bdb10a6b6162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:32:42.007101 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c36053-0383-4302-8753-bdb10a6b6162 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:42.007378 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c36053-0383-4302-8753-bdb10a6b6162 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:42.007651 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c36053-0383-4302-8753-bdb10a6b6162 None admin] PATH_INFO: `/v3/projects/b0db41fa8dcf4888b56f52ba630c67bf` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:42.008830 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c36053-0383-4302-8753-bdb10a6b6162 None admin] RBAC: Authorizing `identity:get_project(project_id=b0db41fa8dcf4888b56f52ba630c67bf)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:32:42.010251 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c36053-0383-4302-8753-bdb10a6b6162 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:32:42.011917 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 607/1199] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jun 16 17:32:41 2022] GET /identity/v3/projects/b0db41fa8dcf4888b56f52ba630c67bf => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:32:42.018733 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f49f61-8845-478b-9d8e-c7e099feaca4 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:32:42.053059 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:32:42.117158 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f49f61-8845-478b-9d8e-c7e099feaca4 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:32:42.152340 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7600c9f2-4d5f-4200-8b5f-36dc83e11843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:32:42.153017 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7600c9f2-4d5f-4200-8b5f-36dc83e11843 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:42.153302 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7600c9f2-4d5f-4200-8b5f-36dc83e11843 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:42.153577 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7600c9f2-4d5f-4200-8b5f-36dc83e11843 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:42.154399 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7600c9f2-4d5f-4200-8b5f-36dc83e11843 None admin] RBAC: Adding query filter params (role.id=3ca2dcbbd29d4a31a3ab78f2299bc239, scope.project.id=b0db41fa8dcf4888b56f52ba630c67bf, user.id=f056650f6e1049cab19652c6818ee974) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:32:42.154709 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7600c9f2-4d5f-4200-8b5f-36dc83e11843 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:32:42.157186 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7600c9f2-4d5f-4200-8b5f-36dc83e11843 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:32:42.162934 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 593/1200] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jun 16 17:32:42 2022] GET /identity/v3/role_assignments?user.id=f056650f6e1049cab19652c6818ee974&scope.project.id=b0db41fa8dcf4888b56f52ba630c67bf&role.id=3ca2dcbbd29d4a31a3ab78f2299bc239 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:32:52.002240 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=66 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:32:52.160264 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=50 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:33:39.257789 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63941e54-2982-4f4b-9cef-29466d515106 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:33:39.257789 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63941e54-2982-4f4b-9cef-29466d515106 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:33:39.260656 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63941e54-2982-4f4b-9cef-29466d515106 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:33:39.261077 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 608/1201] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:33:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:33:39.269481 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311f1b8b-a3e4-4ad1-af6b-f46036d58af6 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:33:39.269985 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311f1b8b-a3e4-4ad1-af6b-f46036d58af6 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:33:39.270213 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311f1b8b-a3e4-4ad1-af6b-f46036d58af6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:33:39.288996 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-311f1b8b-a3e4-4ad1-af6b-f46036d58af6 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:33:39.292425 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-311f1b8b-a3e4-4ad1-af6b-f46036d58af6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:33:39.392697 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 594/1202] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:33:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:33:39.418253 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63941e54-2982-4f4b-9cef-29466d515106 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:33:39.562639 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63941e54-2982-4f4b-9cef-29466d515106 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:33:39.627157 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabaceea-d026-4f1d-b5fb-08e472bcbc4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:33:39.627951 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabaceea-d026-4f1d-b5fb-08e472bcbc4b None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:33:39.628455 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabaceea-d026-4f1d-b5fb-08e472bcbc4b None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:33:39.628612 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabaceea-d026-4f1d-b5fb-08e472bcbc4b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:33:39.641051 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabaceea-d026-4f1d-b5fb-08e472bcbc4b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:33:39.642911 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabaceea-d026-4f1d-b5fb-08e472bcbc4b None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:33:39.665218 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-cabaceea-d026-4f1d-b5fb-08e472bcbc4b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:33:39.665218 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:33:39.665218 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:33:39.665218 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:33:39.665218 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:33:39.665218 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:33:39.665218 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:33:39.665218 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:33:39.665218 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:33:39.665218 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:33:39.665218 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:33:39.665218 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:33:39.665218 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:33:39.665218 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:33:39.665218 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:33:39.666875 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:33:39.666875 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:33:39.666875 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:33:39.666875 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:33:39.666875 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:33:39.666875 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:33:39.666875 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:33:39.666875 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:33:39.666875 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:33:39.666875 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:33:39.666875 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:33:39.666875 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:33:39.666875 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:33:39.666875 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:33:39.666875 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:33:39.666875 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:33:39.666875 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:33:39.668637 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:33:39.668637 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:33:39.668637 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:33:39.668637 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:33:39.668637 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 609/1203] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jun 16 17:33:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:33:39.673308 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311f1b8b-a3e4-4ad1-af6b-f46036d58af6 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:33:39.798773 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311f1b8b-a3e4-4ad1-af6b-f46036d58af6 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:33:39.844507 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:33:39.849121 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cea0a13-54d3-42b6-b787-b097aba9e5c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:33:39.849802 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cea0a13-54d3-42b6-b787-b097aba9e5c8 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:33:39.850135 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cea0a13-54d3-42b6-b787-b097aba9e5c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:33:39.850479 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cea0a13-54d3-42b6-b787-b097aba9e5c8 None admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:33:39.851183 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cea0a13-54d3-42b6-b787-b097aba9e5c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:33:39.851515 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cea0a13-54d3-42b6-b787-b097aba9e5c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:33:39.852937 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cea0a13-54d3-42b6-b787-b097aba9e5c8 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:33:39.867413 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 595/1204] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jun 16 17:33:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:33:39.873469 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabaceea-d026-4f1d-b5fb-08e472bcbc4b None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:33:39.946783 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:33:40.017212 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabaceea-d026-4f1d-b5fb-08e472bcbc4b None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:33:40.073565 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f7879d-d8dd-4f8c-9c58-71133715ea95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:33:40.074525 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f7879d-d8dd-4f8c-9c58-71133715ea95 None admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:33:40.074919 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f7879d-d8dd-4f8c-9c58-71133715ea95 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:33:40.075519 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f7879d-d8dd-4f8c-9c58-71133715ea95 None admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:33:40.076754 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f7879d-d8dd-4f8c-9c58-71133715ea95 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:33:40.079021 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f7879d-d8dd-4f8c-9c58-71133715ea95 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:33:40.103568 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 610/1205] 10.222.0.42 () {60 vars in 1132 bytes} [Thu Jun 16 17:33:39 2022] POST /identity/v3/users => generated 251 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:33:42.940257 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1f66c6-c9f6-40e5-b953-1ffdd9422560 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:33:42.940844 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1f66c6-c9f6-40e5-b953-1ffdd9422560 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:33:42.940927 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1f66c6-c9f6-40e5-b953-1ffdd9422560 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:33:42.942900 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 596/1206] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:33:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:33:42.948086 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae16f3a-5552-4b7e-b0b2-cc05d1759ab5 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:33:42.948532 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae16f3a-5552-4b7e-b0b2-cc05d1759ab5 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:33:42.948709 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae16f3a-5552-4b7e-b0b2-cc05d1759ab5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:33:42.971229 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-dae16f3a-5552-4b7e-b0b2-cc05d1759ab5 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:33:42.973777 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-dae16f3a-5552-4b7e-b0b2-cc05d1759ab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:33:43.054972 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 611/1207] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:33:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:33:43.082195 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1f66c6-c9f6-40e5-b953-1ffdd9422560 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:33:43.211765 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1f66c6-c9f6-40e5-b953-1ffdd9422560 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:33:43.259988 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ef9805-4340-4466-89ed-ad104dda9e2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:33:43.261195 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ef9805-4340-4466-89ed-ad104dda9e2e None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:33:43.261551 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ef9805-4340-4466-89ed-ad104dda9e2e None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:33:43.261956 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ef9805-4340-4466-89ed-ad104dda9e2e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:33:43.283371 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ef9805-4340-4466-89ed-ad104dda9e2e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:33:43.291651 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-b7ef9805-4340-4466-89ed-ad104dda9e2e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:33:43.291651 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:33:43.291651 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:33:43.291651 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:33:43.291651 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:33:43.291651 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:33:43.291651 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:33:43.291651 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:33:43.291651 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:33:43.291651 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:33:43.291651 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:33:43.291651 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:33:43.291651 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:33:43.291651 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:33:43.291651 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:33:43.291651 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:33:43.292813 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:33:43.292813 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:33:43.292813 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:33:43.292813 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:33:43.292813 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:33:43.292813 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:33:43.292813 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:33:43.292813 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:33:43.292813 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:33:43.292813 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:33:43.292813 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:33:43.292813 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:33:43.292813 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:33:43.292813 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:33:43.292813 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:33:43.292813 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:33:43.293687 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:33:43.293687 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:33:43.293687 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:33:43.293687 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:33:43.293687 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:33:43.293687 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:33:43.293687 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:33:43.293687 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:33:43.293687 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:33:43.293687 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:33:43.293687 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:33:43.293687 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:33:43.293687 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 597/1208] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Jun 16 17:33:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:33:43.302253 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae16f3a-5552-4b7e-b0b2-cc05d1759ab5 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:33:43.354362 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:33:43.446362 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae16f3a-5552-4b7e-b0b2-cc05d1759ab5 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:33:43.490785 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90e9ace-bd5e-4afe-8cb1-815158214ae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:33:43.491478 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90e9ace-bd5e-4afe-8cb1-815158214ae2 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:33:43.491762 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90e9ace-bd5e-4afe-8cb1-815158214ae2 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:33:43.492058 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90e9ace-bd5e-4afe-8cb1-815158214ae2 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:33:43.492826 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90e9ace-bd5e-4afe-8cb1-815158214ae2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:33:43.493138 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90e9ace-bd5e-4afe-8cb1-815158214ae2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:33:43.494632 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90e9ace-bd5e-4afe-8cb1-815158214ae2 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:33:43.505096 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 612/1209] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Jun 16 17:33:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:33:43.511651 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ef9805-4340-4466-89ed-ad104dda9e2e None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:33:43.582807 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:33:43.660272 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ef9805-4340-4466-89ed-ad104dda9e2e None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:33:43.705000 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b683aaef-5983-4587-bcd5-2cfb7bb20d30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:33:43.705958 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b683aaef-5983-4587-bcd5-2cfb7bb20d30 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:33:43.706324 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b683aaef-5983-4587-bcd5-2cfb7bb20d30 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:33:43.706632 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b683aaef-5983-4587-bcd5-2cfb7bb20d30 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:33:43.718605 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b683aaef-5983-4587-bcd5-2cfb7bb20d30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:33:43.720154 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b683aaef-5983-4587-bcd5-2cfb7bb20d30 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:33:43.740655 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-b683aaef-5983-4587-bcd5-2cfb7bb20d30 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:33:43.740655 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:33:43.740655 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:33:43.740655 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:33:43.740655 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:33:43.740655 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:33:43.740655 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:33:43.740655 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:33:43.740655 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:33:43.740655 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:33:43.740655 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:33:43.740655 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:33:43.740655 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:33:43.740655 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:33:43.740655 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:33:43.741835 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:33:43.741835 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:33:43.741835 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:33:43.741835 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:33:43.741835 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:33:43.741835 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:33:43.741835 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:33:43.741835 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:33:43.741835 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:33:43.741835 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:33:43.741835 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:33:43.741835 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:33:43.741835 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:33:43.741835 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:33:43.741835 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:33:43.741835 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:33:43.741835 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:33:43.742782 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:33:43.742782 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:33:43.742782 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:33:43.742782 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:33:43.742782 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 598/1210] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jun 16 17:33:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:33:43.748029 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90e9ace-bd5e-4afe-8cb1-815158214ae2 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:33:43.870620 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:33:43.876307 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90e9ace-bd5e-4afe-8cb1-815158214ae2 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:33:43.923099 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a4ea1f-4c8e-4459-b895-185d28c7cf0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:33:43.923883 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a4ea1f-4c8e-4459-b895-185d28c7cf0f None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:33:43.924258 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a4ea1f-4c8e-4459-b895-185d28c7cf0f None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:33:43.924573 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a4ea1f-4c8e-4459-b895-185d28c7cf0f None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:33:43.925511 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a4ea1f-4c8e-4459-b895-185d28c7cf0f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:33:43.925973 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a4ea1f-4c8e-4459-b895-185d28c7cf0f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:33:43.927536 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a4ea1f-4c8e-4459-b895-185d28c7cf0f None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:33:43.951423 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 613/1211] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jun 16 17:33:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:33:43.952086 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b683aaef-5983-4587-bcd5-2cfb7bb20d30 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:33:44.075730 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b683aaef-5983-4587-bcd5-2cfb7bb20d30 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:33:44.077925 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:33:44.113506 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afbafdd-7715-4609-8958-1867bdaafc41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:33:44.114310 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afbafdd-7715-4609-8958-1867bdaafc41 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:33:44.114596 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afbafdd-7715-4609-8958-1867bdaafc41 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:33:44.114879 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afbafdd-7715-4609-8958-1867bdaafc41 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:33:44.122802 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afbafdd-7715-4609-8958-1867bdaafc41 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:33:44.124467 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afbafdd-7715-4609-8958-1867bdaafc41 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:33:44.138927 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-4afbafdd-7715-4609-8958-1867bdaafc41 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 16 17:33:44.138927 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:33:44.138927 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:33:44.138927 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:33:44.138927 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:33:44.138927 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:33:44.138927 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:33:44.138927 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:33:44.138927 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:33:44.138927 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:33:44.138927 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:33:44.138927 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:33:44.138927 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:33:44.138927 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:33:44.138927 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:33:44.138927 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:33:44.139936 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:33:44.139936 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:33:44.139936 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:33:44.139936 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:33:44.139936 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:33:44.139936 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:33:44.139936 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:33:44.139936 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:33:44.139936 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:33:44.139936 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:33:44.139936 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:33:44.139936 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:33:44.139936 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:33:44.139936 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:33:44.139936 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:33:44.139936 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:33:44.139936 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:33:44.139936 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:33:44.141302 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:33:44.141302 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:33:44.141302 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:33:44.141302 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:33:44.141302 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:33:44.141302 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:33:44.141302 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:33:44.141302 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:33:44.141302 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 16 17:33:44.141302 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:33:44.144777 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 599/1212] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jun 16 17:33:43 2022] GET /identity/v3/users/placement => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:33:44.146070 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a4ea1f-4c8e-4459-b895-185d28c7cf0f None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:33:44.262478 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a4ea1f-4c8e-4459-b895-185d28c7cf0f None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:33:44.303026 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166f7297-a823-4fe1-802f-dbab591849b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:33:44.303708 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166f7297-a823-4fe1-802f-dbab591849b2 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:33:44.304078 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166f7297-a823-4fe1-802f-dbab591849b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:33:44.304356 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166f7297-a823-4fe1-802f-dbab591849b2 None admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:33:44.305334 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166f7297-a823-4fe1-802f-dbab591849b2 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:33:44.305733 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166f7297-a823-4fe1-802f-dbab591849b2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:33:44.307374 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166f7297-a823-4fe1-802f-dbab591849b2 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:33:44.330197 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 614/1213] 10.222.0.42 () {56 vars in 1143 bytes} [Thu Jun 16 17:33:44 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:33:44.336034 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afbafdd-7715-4609-8958-1867bdaafc41 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:33:44.461963 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afbafdd-7715-4609-8958-1867bdaafc41 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:33:44.509649 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d9b2a2-fc60-4b31-80fd-aef045584075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:33:44.510493 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d9b2a2-fc60-4b31-80fd-aef045584075 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:33:44.511653 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d9b2a2-fc60-4b31-80fd-aef045584075 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:33:44.511983 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d9b2a2-fc60-4b31-80fd-aef045584075 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:33:44.551742 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:33:44.593748 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d9b2a2-fc60-4b31-80fd-aef045584075 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:33:44.595130 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d9b2a2-fc60-4b31-80fd-aef045584075 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:33:44.721301 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 600/1214] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jun 16 17:33:44 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 17:33:44.727391 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166f7297-a823-4fe1-802f-dbab591849b2 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:33:44.750615 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:33:44.849845 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166f7297-a823-4fe1-802f-dbab591849b2 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:33:44.895640 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479dff88-e0ed-4c5f-8b4b-18cb6e354557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:33:44.896579 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479dff88-e0ed-4c5f-8b4b-18cb6e354557 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:33:44.896975 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479dff88-e0ed-4c5f-8b4b-18cb6e354557 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:33:44.897338 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479dff88-e0ed-4c5f-8b4b-18cb6e354557 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:33:44.911424 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479dff88-e0ed-4c5f-8b4b-18cb6e354557 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:33:44.913044 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479dff88-e0ed-4c5f-8b4b-18cb6e354557 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:33:44.939420 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-479dff88-e0ed-4c5f-8b4b-18cb6e354557 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:33:44.939420 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:33:44.939420 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:33:44.939420 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:33:44.939420 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:33:44.939420 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:33:44.939420 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:33:44.939420 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:33:44.939420 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:33:44.939420 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:33:44.939420 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:33:44.939420 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:33:44.939420 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:33:44.939420 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:33:44.939420 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:33:44.940414 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:33:44.940414 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:33:44.940414 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:33:44.940414 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:33:44.940414 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:33:44.940414 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:33:44.940414 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:33:44.940414 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:33:44.940414 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:33:44.940414 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:33:44.940414 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:33:44.940414 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:33:44.940414 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:33:44.940414 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:33:44.940414 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:33:44.940414 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:33:44.940414 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:33:44.945945 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:33:44.945945 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:33:44.945945 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:33:44.945945 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:33:44.945945 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 615/1215] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jun 16 17:33:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:33:44.946441 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d9b2a2-fc60-4b31-80fd-aef045584075 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:33:45.027687 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:33:45.073236 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d9b2a2-fc60-4b31-80fd-aef045584075 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:33:45.113049 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd49d98-beb0-4294-93d6-eb88e782e367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:33:45.113636 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd49d98-beb0-4294-93d6-eb88e782e367 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:33:45.113915 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd49d98-beb0-4294-93d6-eb88e782e367 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:33:45.114186 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd49d98-beb0-4294-93d6-eb88e782e367 None admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:33:45.115785 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd49d98-beb0-4294-93d6-eb88e782e367 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:33:45.115785 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd49d98-beb0-4294-93d6-eb88e782e367 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:33:45.116562 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd49d98-beb0-4294-93d6-eb88e782e367 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:33:45.135088 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 601/1216] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jun 16 17:33:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:33:45.136614 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479dff88-e0ed-4c5f-8b4b-18cb6e354557 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:33:45.223843 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:33:45.263219 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479dff88-e0ed-4c5f-8b4b-18cb6e354557 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:33:45.305088 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3a1646-15e4-4ee3-af2d-3d658cc8ba0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:33:45.305903 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3a1646-15e4-4ee3-af2d-3d658cc8ba0a None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:33:45.306255 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3a1646-15e4-4ee3-af2d-3d658cc8ba0a None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:33:45.306573 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3a1646-15e4-4ee3-af2d-3d658cc8ba0a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:33:45.314671 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3a1646-15e4-4ee3-af2d-3d658cc8ba0a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:33:45.316491 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3a1646-15e4-4ee3-af2d-3d658cc8ba0a None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:33:45.333730 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-cd3a1646-15e4-4ee3-af2d-3d658cc8ba0a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:33:45.333730 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:33:45.333730 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:33:45.333730 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:33:45.333730 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:33:45.333730 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:33:45.333730 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:33:45.333730 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:33:45.333730 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:33:45.333730 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:33:45.333730 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:33:45.333730 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:33:45.333730 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:33:45.333730 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:33:45.333730 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:33:45.335060 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:33:45.335060 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:33:45.335060 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:33:45.335060 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:33:45.335060 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:33:45.335060 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:33:45.335060 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:33:45.335060 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:33:45.335060 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:33:45.335060 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:33:45.335060 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:33:45.335060 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:33:45.335060 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:33:45.335060 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:33:45.335060 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:33:45.335060 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:33:45.335060 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:33:45.336286 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:33:45.336286 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:33:45.336286 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:33:45.336286 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:33:45.336286 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:33:45.336286 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:33:45.336286 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:33:45.336286 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:33:45.336286 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 616/1217] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jun 16 17:33:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:33:45.341511 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd49d98-beb0-4294-93d6-eb88e782e367 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:33:45.489508 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd49d98-beb0-4294-93d6-eb88e782e367 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:33:45.518207 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:33:45.535044 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6412bd27-adf5-4d80-ae9e-52b4ca39140d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:33:45.535823 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6412bd27-adf5-4d80-ae9e-52b4ca39140d None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:33:45.536138 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6412bd27-adf5-4d80-ae9e-52b4ca39140d None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:33:45.536461 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6412bd27-adf5-4d80-ae9e-52b4ca39140d None admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:33:45.537250 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6412bd27-adf5-4d80-ae9e-52b4ca39140d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:33:45.537553 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6412bd27-adf5-4d80-ae9e-52b4ca39140d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:33:45.538977 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6412bd27-adf5-4d80-ae9e-52b4ca39140d None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:33:45.556601 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 602/1218] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Jun 16 17:33:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:33:45.564489 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3a1646-15e4-4ee3-af2d-3d658cc8ba0a None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:33:45.703199 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3a1646-15e4-4ee3-af2d-3d658cc8ba0a None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:33:45.739638 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:33:45.747564 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac7b1fc-c621-44fc-9eb8-7f4c93d8a291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:33:45.748807 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac7b1fc-c621-44fc-9eb8-7f4c93d8a291 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:33:45.748807 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac7b1fc-c621-44fc-9eb8-7f4c93d8a291 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:33:45.749019 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac7b1fc-c621-44fc-9eb8-7f4c93d8a291 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:33:45.750419 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac7b1fc-c621-44fc-9eb8-7f4c93d8a291 None admin] RBAC: Adding query filter params (role.id=cd7dd29fcdc844c78544402dc30986fb, scope.project.id=8abb87f2f141451a8c6ed832db82be18, user.id=0570d8d9508d4546ae897276f7d181d2) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:33:45.750548 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac7b1fc-c621-44fc-9eb8-7f4c93d8a291 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:33:45.753181 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac7b1fc-c621-44fc-9eb8-7f4c93d8a291 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:33:45.772428 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 617/1219] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jun 16 17:33:45 2022] GET /identity/v3/role_assignments?user.id=0570d8d9508d4546ae897276f7d181d2&scope.project.id=8abb87f2f141451a8c6ed832db82be18&role.id=cd7dd29fcdc844c78544402dc30986fb => generated 259 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:33:48.173457 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dff429e-6602-45a3-b175-8e8008bd83c4 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:33:48.174033 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dff429e-6602-45a3-b175-8e8008bd83c4 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:33:48.174033 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dff429e-6602-45a3-b175-8e8008bd83c4 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:33:48.175886 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 603/1220] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:33:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:33:48.181542 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55858e72-fa9b-47e7-b118-29e54f7057df None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:33:48.181964 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55858e72-fa9b-47e7-b118-29e54f7057df None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:33:48.182174 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55858e72-fa9b-47e7-b118-29e54f7057df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:33:48.200435 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-55858e72-fa9b-47e7-b118-29e54f7057df None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:33:48.202326 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-55858e72-fa9b-47e7-b118-29e54f7057df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:33:48.282600 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 618/1221] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:33:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:33:48.306035 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dff429e-6602-45a3-b175-8e8008bd83c4 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:33:48.427338 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dff429e-6602-45a3-b175-8e8008bd83c4 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:33:48.466182 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9a104a-25a3-4453-a57c-09d9bbe8965b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:33:48.467714 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9a104a-25a3-4453-a57c-09d9bbe8965b None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:33:48.468044 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9a104a-25a3-4453-a57c-09d9bbe8965b None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:33:48.468492 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9a104a-25a3-4453-a57c-09d9bbe8965b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:33:48.481162 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9a104a-25a3-4453-a57c-09d9bbe8965b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:33:48.489736 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-af9a104a-25a3-4453-a57c-09d9bbe8965b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:33:48.489736 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:33:48.489736 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:33:48.489736 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:33:48.489736 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:33:48.489736 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:33:48.489736 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:33:48.489736 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:33:48.489736 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:33:48.489736 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:33:48.489736 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:33:48.489736 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:33:48.489736 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:33:48.489736 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:33:48.489736 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:33:48.489736 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:33:48.490986 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:33:48.490986 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:33:48.490986 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:33:48.490986 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:33:48.490986 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:33:48.490986 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:33:48.490986 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:33:48.490986 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:33:48.490986 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:33:48.490986 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:33:48.490986 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:33:48.490986 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:33:48.490986 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:33:48.490986 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:33:48.490986 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:33:48.490986 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:33:48.492079 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:33:48.492079 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:33:48.492079 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:33:48.492079 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:33:48.492079 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:33:48.492079 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:33:48.492079 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:33:48.492079 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:33:48.492079 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:33:48.492079 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:33:48.492079 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:33:48.492079 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:33:48.492079 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 604/1222] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Jun 16 17:33:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:33:48.497747 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55858e72-fa9b-47e7-b118-29e54f7057df None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:33:48.609815 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55858e72-fa9b-47e7-b118-29e54f7057df None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:33:48.641610 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:33:48.654534 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9240586f-2510-4aab-b620-00e7e46cc32f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:33:48.655201 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9240586f-2510-4aab-b620-00e7e46cc32f None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:33:48.655479 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9240586f-2510-4aab-b620-00e7e46cc32f None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:33:48.655752 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9240586f-2510-4aab-b620-00e7e46cc32f None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:33:48.656582 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9240586f-2510-4aab-b620-00e7e46cc32f None admin] RBAC: Adding query filter params (name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:33:48.656886 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9240586f-2510-4aab-b620-00e7e46cc32f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:33:48.658387 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9240586f-2510-4aab-b620-00e7e46cc32f None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:33:48.667242 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 619/1223] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Jun 16 17:33:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:33:48.674781 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9a104a-25a3-4453-a57c-09d9bbe8965b None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:33:48.722671 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:33:48.783508 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9a104a-25a3-4453-a57c-09d9bbe8965b None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:33:48.818259 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc6ebf4-95a3-4da5-9b42-62be833ddd2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:33:48.819044 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc6ebf4-95a3-4da5-9b42-62be833ddd2d None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:33:48.819331 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc6ebf4-95a3-4da5-9b42-62be833ddd2d None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:33:48.819607 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc6ebf4-95a3-4da5-9b42-62be833ddd2d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:33:48.831163 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc6ebf4-95a3-4da5-9b42-62be833ddd2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:33:48.832636 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc6ebf4-95a3-4da5-9b42-62be833ddd2d None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:33:48.850031 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-adc6ebf4-95a3-4da5-9b42-62be833ddd2d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:33:48.850031 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:33:48.850031 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:33:48.850031 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:33:48.850031 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:33:48.850031 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:33:48.850031 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:33:48.850031 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:33:48.850031 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:33:48.850031 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:33:48.850031 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:33:48.850031 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:33:48.850031 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:33:48.850031 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:33:48.850031 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:33:48.851162 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:33:48.851162 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:33:48.851162 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:33:48.851162 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:33:48.851162 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:33:48.851162 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:33:48.851162 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:33:48.851162 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:33:48.851162 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:33:48.851162 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:33:48.851162 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:33:48.851162 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:33:48.851162 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:33:48.851162 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:33:48.851162 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:33:48.851162 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:33:48.851162 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:33:48.852244 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:33:48.852244 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:33:48.852244 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:33:48.852244 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:33:48.852244 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 605/1224] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jun 16 17:33:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:33:48.857080 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9240586f-2510-4aab-b620-00e7e46cc32f None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:33:48.978454 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9240586f-2510-4aab-b620-00e7e46cc32f None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:33:49.018095 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9711bcc-2b39-4cb0-a260-41cfd60e9693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:33:49.018764 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9711bcc-2b39-4cb0-a260-41cfd60e9693 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:33:49.019049 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9711bcc-2b39-4cb0-a260-41cfd60e9693 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:33:49.019330 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9711bcc-2b39-4cb0-a260-41cfd60e9693 None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:33:49.020301 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9711bcc-2b39-4cb0-a260-41cfd60e9693 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:33:49.020644 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9711bcc-2b39-4cb0-a260-41cfd60e9693 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:33:49.022020 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9711bcc-2b39-4cb0-a260-41cfd60e9693 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:33:49.035513 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 620/1225] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jun 16 17:33:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:33:49.041659 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc6ebf4-95a3-4da5-9b42-62be833ddd2d None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:33:49.148130 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:33:49.155551 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc6ebf4-95a3-4da5-9b42-62be833ddd2d None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:33:49.200834 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffff2271-216a-4e4b-a8a1-ac6e52d53f4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:33:49.202119 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffff2271-216a-4e4b-a8a1-ac6e52d53f4a None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:33:49.202259 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffff2271-216a-4e4b-a8a1-ac6e52d53f4a None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:33:49.202661 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffff2271-216a-4e4b-a8a1-ac6e52d53f4a None admin] PATH_INFO: `/v3/users/placement` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:33:49.211947 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffff2271-216a-4e4b-a8a1-ac6e52d53f4a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:33:49.213853 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffff2271-216a-4e4b-a8a1-ac6e52d53f4a None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:33:49.229961 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-ffff2271-216a-4e4b-a8a1-ac6e52d53f4a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 16 17:33:49.229961 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:33:49.229961 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:33:49.229961 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:33:49.229961 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:33:49.229961 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:33:49.229961 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:33:49.229961 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:33:49.229961 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:33:49.229961 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:33:49.229961 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:33:49.229961 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:33:49.229961 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:33:49.229961 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:33:49.229961 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:33:49.229961 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:33:49.231089 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:33:49.231089 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:33:49.231089 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:33:49.231089 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:33:49.231089 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:33:49.231089 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:33:49.231089 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:33:49.231089 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:33:49.231089 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:33:49.231089 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:33:49.231089 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:33:49.231089 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:33:49.231089 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:33:49.231089 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:33:49.231089 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:33:49.231089 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:33:49.231089 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:33:49.231089 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:33:49.232046 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:33:49.232046 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:33:49.232046 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:33:49.232046 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:33:49.232046 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:33:49.232046 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:33:49.232046 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:33:49.232046 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:33:49.232046 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 16 17:33:49.232046 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:33:49.232046 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 606/1226] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jun 16 17:33:49 2022] GET /identity/v3/users/placement => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:33:49.237898 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9711bcc-2b39-4cb0-a260-41cfd60e9693 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:33:49.296082 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:33:49.353470 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9711bcc-2b39-4cb0-a260-41cfd60e9693 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:33:49.402230 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d788109b-5892-463f-9c89-58f5e9060fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:33:49.403198 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d788109b-5892-463f-9c89-58f5e9060fe3 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:33:49.403514 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d788109b-5892-463f-9c89-58f5e9060fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:33:49.403830 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d788109b-5892-463f-9c89-58f5e9060fe3 None admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:33:49.405301 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d788109b-5892-463f-9c89-58f5e9060fe3 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:33:49.405615 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d788109b-5892-463f-9c89-58f5e9060fe3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:33:49.407221 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d788109b-5892-463f-9c89-58f5e9060fe3 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:33:49.426732 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 621/1227] 10.222.0.42 () {56 vars in 1143 bytes} [Thu Jun 16 17:33:49 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:33:49.432717 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffff2271-216a-4e4b-a8a1-ac6e52d53f4a None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:33:49.542531 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffff2271-216a-4e4b-a8a1-ac6e52d53f4a None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:33:49.581509 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45e4680-ef11-4ae7-a30c-5afea3e22cf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:33:49.582305 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45e4680-ef11-4ae7-a30c-5afea3e22cf7 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:33:49.582592 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45e4680-ef11-4ae7-a30c-5afea3e22cf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:33:49.582866 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45e4680-ef11-4ae7-a30c-5afea3e22cf7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:33:49.595468 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45e4680-ef11-4ae7-a30c-5afea3e22cf7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:33:49.597162 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45e4680-ef11-4ae7-a30c-5afea3e22cf7 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:33:49.618999 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-e45e4680-ef11-4ae7-a30c-5afea3e22cf7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:33:49.618999 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:33:49.618999 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:33:49.618999 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:33:49.618999 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:33:49.618999 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:33:49.618999 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:33:49.618999 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:33:49.618999 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:33:49.618999 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:33:49.618999 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:33:49.618999 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:33:49.618999 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:33:49.618999 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:33:49.618999 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:33:49.625630 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:33:49.625630 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:33:49.625630 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:33:49.625630 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:33:49.625630 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:33:49.625630 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:33:49.625630 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:33:49.625630 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:33:49.625630 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:33:49.625630 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:33:49.625630 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:33:49.625630 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:33:49.625630 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:33:49.625630 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:33:49.625630 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:33:49.625630 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:33:49.625630 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:33:49.626813 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:33:49.626813 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:33:49.626813 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:33:49.626813 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:33:49.626813 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 607/1228] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jun 16 17:33:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:33:49.627892 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d788109b-5892-463f-9c89-58f5e9060fe3 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:33:49.747690 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d788109b-5892-463f-9c89-58f5e9060fe3 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:33:49.750431 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:33:49.788773 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17885f5-7859-4a3e-a82b-682417bf5616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:33:49.789550 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17885f5-7859-4a3e-a82b-682417bf5616 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:33:49.789912 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17885f5-7859-4a3e-a82b-682417bf5616 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:33:49.790212 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17885f5-7859-4a3e-a82b-682417bf5616 None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:33:49.791093 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17885f5-7859-4a3e-a82b-682417bf5616 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:33:49.791467 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17885f5-7859-4a3e-a82b-682417bf5616 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:33:49.793251 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17885f5-7859-4a3e-a82b-682417bf5616 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:33:49.806749 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 622/1229] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jun 16 17:33:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:33:49.812703 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45e4680-ef11-4ae7-a30c-5afea3e22cf7 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:33:49.821946 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:33:49.940282 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45e4680-ef11-4ae7-a30c-5afea3e22cf7 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:33:49.978407 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908718eb-b9ff-41af-b478-648117d76a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:33:49.979229 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908718eb-b9ff-41af-b478-648117d76a88 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:33:49.979515 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908718eb-b9ff-41af-b478-648117d76a88 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:33:49.979793 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908718eb-b9ff-41af-b478-648117d76a88 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:33:49.987444 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908718eb-b9ff-41af-b478-648117d76a88 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:33:49.989058 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908718eb-b9ff-41af-b478-648117d76a88 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:33:50.004434 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-908718eb-b9ff-41af-b478-648117d76a88 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:33:50.004434 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:33:50.004434 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:33:50.004434 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:33:50.004434 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:33:50.004434 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:33:50.004434 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:33:50.004434 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:33:50.004434 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:33:50.004434 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:33:50.004434 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:33:50.004434 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:33:50.004434 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:33:50.004434 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:33:50.004434 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:33:50.005598 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:33:50.005598 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:33:50.005598 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:33:50.005598 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:33:50.005598 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:33:50.005598 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:33:50.005598 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:33:50.005598 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:33:50.005598 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:33:50.005598 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:33:50.005598 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:33:50.005598 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:33:50.005598 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:33:50.005598 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:33:50.005598 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:33:50.005598 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:33:50.005598 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:33:50.006491 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:33:50.006491 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:33:50.006491 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:33:50.006491 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:33:50.006491 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:33:50.006491 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:33:50.006491 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:33:50.006491 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:33:50.006491 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 608/1230] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jun 16 17:33:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:33:50.011630 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17885f5-7859-4a3e-a82b-682417bf5616 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:33:50.126461 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17885f5-7859-4a3e-a82b-682417bf5616 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:33:50.164715 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40743201-838c-4e2d-b265-ed5e66097462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:33:50.165386 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40743201-838c-4e2d-b265-ed5e66097462 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:33:50.165748 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40743201-838c-4e2d-b265-ed5e66097462 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:33:50.166013 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40743201-838c-4e2d-b265-ed5e66097462 None admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:33:50.166856 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40743201-838c-4e2d-b265-ed5e66097462 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:33:50.167211 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40743201-838c-4e2d-b265-ed5e66097462 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:33:50.168785 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40743201-838c-4e2d-b265-ed5e66097462 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:33:50.181554 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 623/1231] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Jun 16 17:33:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:33:50.188321 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908718eb-b9ff-41af-b478-648117d76a88 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:33:50.266306 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:33:50.305208 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908718eb-b9ff-41af-b478-648117d76a88 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:33:50.342130 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d2f60a-2875-4fe0-9cb5-ed7fd5bd28af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:33:50.343017 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d2f60a-2875-4fe0-9cb5-ed7fd5bd28af None admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:33:50.343298 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d2f60a-2875-4fe0-9cb5-ed7fd5bd28af None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:33:50.343577 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d2f60a-2875-4fe0-9cb5-ed7fd5bd28af None admin] PATH_INFO: `/v3/projects/8abb87f2f141451a8c6ed832db82be18/users/0570d8d9508d4546ae897276f7d181d2/roles/cd7dd29fcdc844c78544402dc30986fb` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:33:50.359626 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d2f60a-2875-4fe0-9cb5-ed7fd5bd28af None admin] RBAC: Authorizing `identity:create_grant(project_id=8abb87f2f141451a8c6ed832db82be18, user_id=0570d8d9508d4546ae897276f7d181d2, role_id=cd7dd29fcdc844c78544402dc30986fb)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:33:50.361112 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d2f60a-2875-4fe0-9cb5-ed7fd5bd28af None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:33:50.375771 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 609/1232] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Jun 16 17:33:50 2022] PUT /identity/v3/projects/8abb87f2f141451a8c6ed832db82be18/users/0570d8d9508d4546ae897276f7d181d2/roles/cd7dd29fcdc844c78544402dc30986fb => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:33:52.800620 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b8cd1f-5f14-4568-a6b0-19122ee9230a None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:33:52.801179 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b8cd1f-5f14-4568-a6b0-19122ee9230a None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:33:52.801179 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b8cd1f-5f14-4568-a6b0-19122ee9230a None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:33:52.802546 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 624/1233] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:33:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:33:52.809745 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13e5171-44c6-43b4-bb63-b6a5e22d7a89 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:33:52.810189 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13e5171-44c6-43b4-bb63-b6a5e22d7a89 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:33:52.810517 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13e5171-44c6-43b4-bb63-b6a5e22d7a89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:33:52.829127 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-b13e5171-44c6-43b4-bb63-b6a5e22d7a89 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:33:52.830337 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-b13e5171-44c6-43b4-bb63-b6a5e22d7a89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:33:52.921371 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 610/1234] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:33:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 17:33:52.948036 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b8cd1f-5f14-4568-a6b0-19122ee9230a None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:33:52.975578 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:33:53.064181 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b8cd1f-5f14-4568-a6b0-19122ee9230a None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:33:53.104469 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceed161-4849-4ba4-98f8-f80504813e5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:33:53.105356 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceed161-4849-4ba4-98f8-f80504813e5b None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:33:53.105707 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceed161-4849-4ba4-98f8-f80504813e5b None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:33:53.105986 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceed161-4849-4ba4-98f8-f80504813e5b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:33:53.117862 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ceed161-4849-4ba4-98f8-f80504813e5b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:33:53.126672 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-3ceed161-4849-4ba4-98f8-f80504813e5b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:33:53.126672 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:33:53.126672 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:33:53.126672 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:33:53.126672 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:33:53.126672 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:33:53.126672 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:33:53.126672 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:33:53.126672 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:33:53.126672 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:33:53.126672 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:33:53.126672 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:33:53.126672 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:33:53.126672 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:33:53.126672 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:33:53.126672 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:33:53.127866 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:33:53.127866 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:33:53.127866 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:33:53.127866 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:33:53.127866 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:33:53.127866 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:33:53.127866 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:33:53.127866 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:33:53.127866 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:33:53.127866 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:33:53.127866 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:33:53.127866 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:33:53.127866 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:33:53.127866 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:33:53.127866 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:33:53.127866 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:33:53.128904 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:33:53.128904 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:33:53.128904 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:33:53.128904 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:33:53.128904 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:33:53.128904 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:33:53.128904 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:33:53.128904 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:33:53.128904 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:33:53.128904 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:33:53.128904 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:33:53.128904 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:33:53.128904 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 625/1235] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Jun 16 17:33:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:33:53.134120 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13e5171-44c6-43b4-bb63-b6a5e22d7a89 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:33:53.185747 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:33:53.257372 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13e5171-44c6-43b4-bb63-b6a5e22d7a89 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:33:53.297016 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3776a7-b863-4885-9e81-b891fc3a5755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:33:53.297742 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3776a7-b863-4885-9e81-b891fc3a5755 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:33:53.298030 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3776a7-b863-4885-9e81-b891fc3a5755 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:33:53.298343 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3776a7-b863-4885-9e81-b891fc3a5755 None admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:33:53.299154 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3776a7-b863-4885-9e81-b891fc3a5755 None admin] RBAC: Adding query filter params (name=service) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:33:53.299467 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3776a7-b863-4885-9e81-b891fc3a5755 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:33:53.301105 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3776a7-b863-4885-9e81-b891fc3a5755 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:33:53.310548 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 611/1236] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Jun 16 17:33:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:33:53.317247 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceed161-4849-4ba4-98f8-f80504813e5b None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:33:53.413855 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:33:53.432315 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceed161-4849-4ba4-98f8-f80504813e5b None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:33:53.469882 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43722302-7771-4b52-a6d1-d4d112662d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:33:53.470689 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43722302-7771-4b52-a6d1-d4d112662d0d None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:33:53.470963 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43722302-7771-4b52-a6d1-d4d112662d0d None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:33:53.471245 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43722302-7771-4b52-a6d1-d4d112662d0d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:33:53.481755 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43722302-7771-4b52-a6d1-d4d112662d0d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:33:53.483454 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43722302-7771-4b52-a6d1-d4d112662d0d None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:33:53.502273 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-43722302-7771-4b52-a6d1-d4d112662d0d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:33:53.502273 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:33:53.502273 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:33:53.502273 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:33:53.502273 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:33:53.502273 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:33:53.502273 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:33:53.502273 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:33:53.502273 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:33:53.502273 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:33:53.502273 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:33:53.502273 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:33:53.502273 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:33:53.502273 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:33:53.502273 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:33:53.503504 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:33:53.503504 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:33:53.503504 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:33:53.503504 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:33:53.503504 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:33:53.503504 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:33:53.503504 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:33:53.503504 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:33:53.503504 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:33:53.503504 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:33:53.503504 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:33:53.503504 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:33:53.503504 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:33:53.503504 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:33:53.503504 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:33:53.503504 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:33:53.503504 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:33:53.504951 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:33:53.504951 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:33:53.504951 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:33:53.504951 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:33:53.504951 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 626/1237] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jun 16 17:33:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:33:53.509398 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3776a7-b863-4885-9e81-b891fc3a5755 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:33:53.620700 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3776a7-b863-4885-9e81-b891fc3a5755 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:33:53.631227 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:33:53.664520 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846a068e-04a7-4e5f-91a0-c579d8a75c04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:33:53.665188 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846a068e-04a7-4e5f-91a0-c579d8a75c04 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:33:53.665474 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846a068e-04a7-4e5f-91a0-c579d8a75c04 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:33:53.665752 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846a068e-04a7-4e5f-91a0-c579d8a75c04 None admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:33:53.666470 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846a068e-04a7-4e5f-91a0-c579d8a75c04 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:33:53.666781 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846a068e-04a7-4e5f-91a0-c579d8a75c04 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:33:53.668149 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846a068e-04a7-4e5f-91a0-c579d8a75c04 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:33:53.680842 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 612/1238] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jun 16 17:33:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:33:53.687287 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43722302-7771-4b52-a6d1-d4d112662d0d None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:33:53.797952 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43722302-7771-4b52-a6d1-d4d112662d0d None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:33:53.835712 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784008b7-64b5-4297-949c-bee09518c3d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:33:53.836763 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784008b7-64b5-4297-949c-bee09518c3d0 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:33:53.837053 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784008b7-64b5-4297-949c-bee09518c3d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:33:53.837335 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784008b7-64b5-4297-949c-bee09518c3d0 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:33:53.840661 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:33:53.846811 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784008b7-64b5-4297-949c-bee09518c3d0 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:33:53.848440 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784008b7-64b5-4297-949c-bee09518c3d0 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:33:53.865203 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-784008b7-64b5-4297-949c-bee09518c3d0 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 16 17:33:53.865203 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:33:53.865203 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:33:53.865203 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:33:53.865203 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:33:53.865203 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:33:53.865203 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:33:53.865203 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:33:53.865203 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:33:53.865203 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:33:53.865203 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:33:53.865203 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:33:53.865203 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:33:53.865203 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:33:53.865203 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:33:53.865203 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:33:53.866392 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:33:53.866392 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:33:53.866392 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:33:53.866392 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:33:53.866392 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:33:53.866392 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:33:53.866392 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:33:53.866392 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:33:53.866392 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:33:53.866392 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:33:53.866392 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:33:53.866392 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:33:53.866392 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:33:53.866392 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:33:53.866392 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:33:53.866392 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:33:53.866392 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:33:53.866392 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:33:53.867532 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:33:53.867532 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:33:53.867532 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:33:53.867532 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:33:53.867532 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:33:53.867532 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:33:53.867532 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:33:53.867532 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:33:53.867532 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 16 17:33:53.867532 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:33:53.867532 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 627/1239] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jun 16 17:33:53 2022] GET /identity/v3/users/placement => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:33:53.873467 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846a068e-04a7-4e5f-91a0-c579d8a75c04 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:33:53.990393 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846a068e-04a7-4e5f-91a0-c579d8a75c04 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:33:54.028354 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23834488-c6a8-46fc-bdc5-474a33dc1d87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:33:54.029685 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23834488-c6a8-46fc-bdc5-474a33dc1d87 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:33:54.029929 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23834488-c6a8-46fc-bdc5-474a33dc1d87 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:33:54.030208 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23834488-c6a8-46fc-bdc5-474a33dc1d87 None admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:33:54.031049 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23834488-c6a8-46fc-bdc5-474a33dc1d87 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:33:54.031359 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23834488-c6a8-46fc-bdc5-474a33dc1d87 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:33:54.033029 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23834488-c6a8-46fc-bdc5-474a33dc1d87 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:33:54.062690 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 613/1240] 10.222.0.42 () {56 vars in 1143 bytes} [Thu Jun 16 17:33:53 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:33:54.063909 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784008b7-64b5-4297-949c-bee09518c3d0 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:33:54.167998 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784008b7-64b5-4297-949c-bee09518c3d0 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:33:54.206598 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb495411-1573-4112-811b-d22f0ba804b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:33:54.207261 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb495411-1573-4112-811b-d22f0ba804b3 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:33:54.207546 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb495411-1573-4112-811b-d22f0ba804b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:33:54.207826 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb495411-1573-4112-811b-d22f0ba804b3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:33:54.273387 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:33:54.282012 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb495411-1573-4112-811b-d22f0ba804b3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:33:54.283810 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb495411-1573-4112-811b-d22f0ba804b3 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:33:54.407219 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 628/1241] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jun 16 17:33:54 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 17:33:54.415485 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23834488-c6a8-46fc-bdc5-474a33dc1d87 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:33:54.452093 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:33:54.536950 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23834488-c6a8-46fc-bdc5-474a33dc1d87 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:33:54.575642 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e412d06e-d78d-45bc-97ae-87134d6a0e3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:33:54.577436 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e412d06e-d78d-45bc-97ae-87134d6a0e3a None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:33:54.577765 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e412d06e-d78d-45bc-97ae-87134d6a0e3a None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:33:54.578090 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e412d06e-d78d-45bc-97ae-87134d6a0e3a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:33:54.589865 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e412d06e-d78d-45bc-97ae-87134d6a0e3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:33:54.591559 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e412d06e-d78d-45bc-97ae-87134d6a0e3a None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:33:54.612570 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-e412d06e-d78d-45bc-97ae-87134d6a0e3a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:33:54.612570 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:33:54.612570 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:33:54.612570 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:33:54.612570 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:33:54.612570 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:33:54.612570 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:33:54.612570 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:33:54.612570 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:33:54.612570 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:33:54.612570 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:33:54.612570 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:33:54.612570 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:33:54.612570 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:33:54.612570 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:33:54.613784 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:33:54.613784 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:33:54.613784 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:33:54.613784 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:33:54.613784 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:33:54.613784 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:33:54.613784 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:33:54.613784 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:33:54.613784 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:33:54.613784 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:33:54.613784 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:33:54.613784 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:33:54.613784 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:33:54.613784 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:33:54.613784 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:33:54.613784 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:33:54.613784 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:33:54.615000 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:33:54.615000 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:33:54.615000 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:33:54.615000 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:33:54.618888 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 614/1242] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jun 16 17:33:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:33:54.620085 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb495411-1573-4112-811b-d22f0ba804b3 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:33:54.724837 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:33:54.732701 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb495411-1573-4112-811b-d22f0ba804b3 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:33:54.778655 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8526e38b-ccc7-49fc-8cb4-f9cbfac7a6d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:33:54.780061 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8526e38b-ccc7-49fc-8cb4-f9cbfac7a6d2 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:33:54.780363 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8526e38b-ccc7-49fc-8cb4-f9cbfac7a6d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:33:54.780691 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8526e38b-ccc7-49fc-8cb4-f9cbfac7a6d2 None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:33:54.781627 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8526e38b-ccc7-49fc-8cb4-f9cbfac7a6d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:33:54.781936 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8526e38b-ccc7-49fc-8cb4-f9cbfac7a6d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:33:54.783533 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8526e38b-ccc7-49fc-8cb4-f9cbfac7a6d2 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:33:54.797115 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 629/1243] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jun 16 17:33:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:33:54.802708 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e412d06e-d78d-45bc-97ae-87134d6a0e3a None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:33:54.909909 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:33:54.938042 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e412d06e-d78d-45bc-97ae-87134d6a0e3a None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:33:54.979761 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa61913-bf13-4079-9453-1494e55ced03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:33:54.980615 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa61913-bf13-4079-9453-1494e55ced03 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:33:54.980940 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa61913-bf13-4079-9453-1494e55ced03 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:33:54.981248 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa61913-bf13-4079-9453-1494e55ced03 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:33:54.989711 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa61913-bf13-4079-9453-1494e55ced03 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:33:54.991615 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa61913-bf13-4079-9453-1494e55ced03 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:33:55.006642 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-bfa61913-bf13-4079-9453-1494e55ced03 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:33:55.006642 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:33:55.006642 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:33:55.006642 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:33:55.006642 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:33:55.006642 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:33:55.006642 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:33:55.006642 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:33:55.006642 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:33:55.006642 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:33:55.006642 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:33:55.006642 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:33:55.006642 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:33:55.006642 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:33:55.006642 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:33:55.007778 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:33:55.007778 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:33:55.007778 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:33:55.007778 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:33:55.007778 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:33:55.007778 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:33:55.007778 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:33:55.007778 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:33:55.007778 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:33:55.007778 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:33:55.007778 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:33:55.007778 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:33:55.007778 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:33:55.007778 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:33:55.007778 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:33:55.007778 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:33:55.007778 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:33:55.008962 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:33:55.008962 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:33:55.008962 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:33:55.008962 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:33:55.008962 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:33:55.008962 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:33:55.008962 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:33:55.008962 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:33:55.008962 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 615/1244] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jun 16 17:33:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:33:55.013873 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8526e38b-ccc7-49fc-8cb4-f9cbfac7a6d2 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:33:55.134173 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8526e38b-ccc7-49fc-8cb4-f9cbfac7a6d2 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:33:55.174827 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c273367d-669e-464a-b945-52405e032b2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:33:55.175529 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c273367d-669e-464a-b945-52405e032b2c None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:33:55.175820 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c273367d-669e-464a-b945-52405e032b2c None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:33:55.176097 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c273367d-669e-464a-b945-52405e032b2c None admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:33:55.177032 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c273367d-669e-464a-b945-52405e032b2c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:33:55.177391 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c273367d-669e-464a-b945-52405e032b2c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:33:55.178995 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c273367d-669e-464a-b945-52405e032b2c None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:33:55.193311 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 630/1245] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Jun 16 17:33:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:33:55.200157 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa61913-bf13-4079-9453-1494e55ced03 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:33:55.326748 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa61913-bf13-4079-9453-1494e55ced03 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:33:55.365460 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14fa92d-a33c-494e-973c-c3aecf98f4c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:33:55.366123 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14fa92d-a33c-494e-973c-c3aecf98f4c8 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:33:55.366407 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14fa92d-a33c-494e-973c-c3aecf98f4c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:33:55.366688 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14fa92d-a33c-494e-973c-c3aecf98f4c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:33:55.367514 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14fa92d-a33c-494e-973c-c3aecf98f4c8 None admin] RBAC: Adding query filter params (role.id=cd7dd29fcdc844c78544402dc30986fb, scope.project.id=8abb87f2f141451a8c6ed832db82be18, user.id=0570d8d9508d4546ae897276f7d181d2) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:33:55.367839 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14fa92d-a33c-494e-973c-c3aecf98f4c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:33:55.369542 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14fa92d-a33c-494e-973c-c3aecf98f4c8 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:33:55.376339 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 616/1246] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jun 16 17:33:55 2022] GET /identity/v3/role_assignments?user.id=0570d8d9508d4546ae897276f7d181d2&scope.project.id=8abb87f2f141451a8c6ed832db82be18&role.id=cd7dd29fcdc844c78544402dc30986fb => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:33:57.802746 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1c3a15-cb9c-47dc-af20-0989e7ebef44 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:33:57.803410 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1c3a15-cb9c-47dc-af20-0989e7ebef44 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:33:57.803410 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1c3a15-cb9c-47dc-af20-0989e7ebef44 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:33:57.804989 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 631/1247] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:33:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:33:57.810942 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc545255-b869-41fe-ade7-e95ead7e2909 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:33:57.812042 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc545255-b869-41fe-ade7-e95ead7e2909 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:33:57.812042 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc545255-b869-41fe-ade7-e95ead7e2909 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:33:57.816979 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:33:57.832508 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-dc545255-b869-41fe-ade7-e95ead7e2909 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:33:57.833826 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-dc545255-b869-41fe-ade7-e95ead7e2909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:33:57.914544 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 617/1248] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:33:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:33:57.941007 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1c3a15-cb9c-47dc-af20-0989e7ebef44 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:33:57.960309 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:33:58.075812 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1c3a15-cb9c-47dc-af20-0989e7ebef44 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:33:58.115188 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8469f651-20c2-4a0c-8aa6-1327c650dcb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:33:58.116153 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8469f651-20c2-4a0c-8aa6-1327c650dcb9 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:33:58.116479 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8469f651-20c2-4a0c-8aa6-1327c650dcb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:33:58.116838 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8469f651-20c2-4a0c-8aa6-1327c650dcb9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:33:58.127648 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8469f651-20c2-4a0c-8aa6-1327c650dcb9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:33:58.135496 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-8469f651-20c2-4a0c-8aa6-1327c650dcb9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:33:58.135496 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:33:58.135496 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:33:58.135496 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:33:58.135496 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:33:58.135496 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:33:58.135496 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:33:58.135496 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:33:58.135496 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:33:58.135496 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:33:58.135496 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:33:58.135496 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:33:58.135496 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:33:58.135496 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:33:58.135496 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:33:58.135496 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:33:58.136730 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:33:58.136730 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:33:58.136730 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:33:58.136730 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:33:58.136730 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:33:58.136730 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:33:58.136730 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:33:58.136730 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:33:58.136730 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:33:58.136730 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:33:58.136730 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:33:58.136730 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:33:58.136730 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:33:58.136730 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:33:58.136730 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:33:58.136730 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:33:58.145837 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:33:58.145837 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:33:58.145837 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:33:58.145837 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:33:58.145837 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:33:58.145837 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:33:58.145837 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:33:58.145837 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:33:58.145837 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:33:58.145837 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:33:58.145837 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:33:58.145837 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:33:58.145837 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 632/1249] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Jun 16 17:33:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:33:58.146980 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc545255-b869-41fe-ade7-e95ead7e2909 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:33:58.258470 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc545255-b869-41fe-ade7-e95ead7e2909 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:33:58.290757 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:33:58.296717 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1396eacf-e8c6-4e66-af29-05381d60fbe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:33:58.297412 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1396eacf-e8c6-4e66-af29-05381d60fbe7 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:33:58.297695 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1396eacf-e8c6-4e66-af29-05381d60fbe7 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:33:58.297978 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1396eacf-e8c6-4e66-af29-05381d60fbe7 None admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:33:58.298820 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1396eacf-e8c6-4e66-af29-05381d60fbe7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:33:58.299135 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1396eacf-e8c6-4e66-af29-05381d60fbe7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:33:58.300690 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1396eacf-e8c6-4e66-af29-05381d60fbe7 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:33:58.309664 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 618/1250] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jun 16 17:33:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:33:58.316507 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8469f651-20c2-4a0c-8aa6-1327c650dcb9 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:33:58.418346 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:33:58.446836 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8469f651-20c2-4a0c-8aa6-1327c650dcb9 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:33:58.483186 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697e3f12-a12d-405e-9190-30955f61db82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:33:58.483991 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697e3f12-a12d-405e-9190-30955f61db82 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:33:58.484283 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697e3f12-a12d-405e-9190-30955f61db82 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:33:58.484595 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697e3f12-a12d-405e-9190-30955f61db82 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:33:58.496548 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697e3f12-a12d-405e-9190-30955f61db82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:33:58.498564 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697e3f12-a12d-405e-9190-30955f61db82 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:33:58.516630 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-697e3f12-a12d-405e-9190-30955f61db82 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:33:58.516630 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:33:58.516630 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:33:58.516630 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:33:58.516630 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:33:58.516630 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:33:58.516630 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:33:58.516630 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:33:58.516630 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:33:58.516630 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:33:58.516630 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:33:58.516630 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:33:58.516630 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:33:58.516630 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:33:58.516630 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:33:58.517704 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:33:58.517704 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:33:58.517704 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:33:58.517704 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:33:58.517704 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:33:58.517704 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:33:58.517704 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:33:58.517704 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:33:58.517704 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:33:58.517704 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:33:58.517704 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:33:58.517704 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:33:58.517704 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:33:58.517704 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:33:58.517704 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:33:58.517704 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:33:58.517704 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:33:58.519287 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:33:58.519287 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:33:58.519287 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:33:58.519287 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:33:58.519287 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 633/1251] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jun 16 17:33:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:33:58.523835 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1396eacf-e8c6-4e66-af29-05381d60fbe7 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:33:58.632943 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1396eacf-e8c6-4e66-af29-05381d60fbe7 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:33:58.668999 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b8e74a-335a-415f-ac81-827ece6e3809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:33:58.669578 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b8e74a-335a-415f-ac81-827ece6e3809 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:33:58.669960 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b8e74a-335a-415f-ac81-827ece6e3809 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:33:58.670261 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b8e74a-335a-415f-ac81-827ece6e3809 None admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:33:58.670983 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b8e74a-335a-415f-ac81-827ece6e3809 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:33:58.671290 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b8e74a-335a-415f-ac81-827ece6e3809 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:33:58.672679 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b8e74a-335a-415f-ac81-827ece6e3809 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:33:58.690751 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 619/1252] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jun 16 17:33:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:33:58.692054 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697e3f12-a12d-405e-9190-30955f61db82 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:33:58.812387 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697e3f12-a12d-405e-9190-30955f61db82 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:33:58.841033 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:33:58.850277 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e215d160-eb50-4364-bcea-8ecfda418a1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:33:58.851105 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e215d160-eb50-4364-bcea-8ecfda418a1c None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:33:58.851383 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e215d160-eb50-4364-bcea-8ecfda418a1c None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:33:58.851656 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e215d160-eb50-4364-bcea-8ecfda418a1c None admin] PATH_INFO: `/v3/users/placement` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:33:58.860905 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e215d160-eb50-4364-bcea-8ecfda418a1c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:33:58.862411 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e215d160-eb50-4364-bcea-8ecfda418a1c None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:33:58.878859 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-e215d160-eb50-4364-bcea-8ecfda418a1c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 16 17:33:58.878859 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:33:58.878859 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:33:58.878859 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:33:58.878859 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:33:58.878859 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:33:58.878859 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:33:58.878859 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:33:58.878859 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:33:58.878859 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:33:58.878859 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:33:58.878859 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:33:58.878859 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:33:58.878859 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:33:58.878859 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:33:58.878859 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:33:58.880133 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:33:58.880133 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:33:58.880133 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:33:58.880133 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:33:58.880133 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:33:58.880133 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:33:58.880133 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:33:58.880133 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:33:58.880133 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:33:58.880133 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:33:58.880133 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:33:58.880133 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:33:58.880133 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:33:58.880133 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:33:58.880133 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:33:58.880133 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:33:58.880133 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:33:58.880133 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:33:58.881325 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:33:58.881325 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:33:58.881325 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:33:58.881325 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:33:58.881325 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:33:58.881325 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:33:58.881325 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:33:58.881325 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:33:58.881325 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 16 17:33:58.881325 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:33:58.881325 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 634/1253] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jun 16 17:33:58 2022] GET /identity/v3/users/placement => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:33:58.886477 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b8e74a-335a-415f-ac81-827ece6e3809 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:33:58.946508 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:33:58.993965 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b8e74a-335a-415f-ac81-827ece6e3809 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:33:59.033016 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbafd826-be83-444f-9d91-30c403cf5478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:33:59.033691 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbafd826-be83-444f-9d91-30c403cf5478 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:33:59.033967 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbafd826-be83-444f-9d91-30c403cf5478 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:33:59.034236 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbafd826-be83-444f-9d91-30c403cf5478 None admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:33:59.035068 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbafd826-be83-444f-9d91-30c403cf5478 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:33:59.035368 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbafd826-be83-444f-9d91-30c403cf5478 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:33:59.036851 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbafd826-be83-444f-9d91-30c403cf5478 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:33:59.061787 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 620/1254] 10.222.0.42 () {56 vars in 1143 bytes} [Thu Jun 16 17:33:58 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:33:59.062368 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e215d160-eb50-4364-bcea-8ecfda418a1c None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:33:59.174773 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e215d160-eb50-4364-bcea-8ecfda418a1c None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:33:59.210395 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57ae50f-86a5-42e2-98b0-df7bc72976fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:33:59.210976 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57ae50f-86a5-42e2-98b0-df7bc72976fb None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:33:59.211255 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57ae50f-86a5-42e2-98b0-df7bc72976fb None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:33:59.211525 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57ae50f-86a5-42e2-98b0-df7bc72976fb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:33:59.267911 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:33:59.285323 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57ae50f-86a5-42e2-98b0-df7bc72976fb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:33:59.286762 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57ae50f-86a5-42e2-98b0-df7bc72976fb None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:33:59.396122 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 635/1255] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jun 16 17:33:59 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 17:33:59.402481 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbafd826-be83-444f-9d91-30c403cf5478 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:33:59.506745 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbafd826-be83-444f-9d91-30c403cf5478 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:33:59.520568 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:33:59.541832 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb605e7-64c9-424f-859e-9a2d9ee63ae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:33:59.542617 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb605e7-64c9-424f-859e-9a2d9ee63ae0 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:33:59.542898 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb605e7-64c9-424f-859e-9a2d9ee63ae0 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:33:59.543170 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb605e7-64c9-424f-859e-9a2d9ee63ae0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:33:59.553045 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb605e7-64c9-424f-859e-9a2d9ee63ae0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:33:59.554515 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb605e7-64c9-424f-859e-9a2d9ee63ae0 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:33:59.571456 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-4cb605e7-64c9-424f-859e-9a2d9ee63ae0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:33:59.571456 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:33:59.571456 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:33:59.571456 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:33:59.571456 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:33:59.571456 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:33:59.571456 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:33:59.571456 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:33:59.571456 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:33:59.571456 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:33:59.571456 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:33:59.571456 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:33:59.571456 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:33:59.571456 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:33:59.571456 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:33:59.572625 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:33:59.572625 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:33:59.572625 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:33:59.572625 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:33:59.572625 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:33:59.572625 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:33:59.572625 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:33:59.572625 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:33:59.572625 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:33:59.572625 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:33:59.572625 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:33:59.572625 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:33:59.572625 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:33:59.572625 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:33:59.572625 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:33:59.572625 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:33:59.572625 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:33:59.573811 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:33:59.573811 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:33:59.573811 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:33:59.573811 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:33:59.573811 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 621/1256] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jun 16 17:33:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:33:59.578861 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57ae50f-86a5-42e2-98b0-df7bc72976fb None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:33:59.664147 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:33:59.676108 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57ae50f-86a5-42e2-98b0-df7bc72976fb None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:33:59.709946 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695403d9-1713-49d3-bee8-39c47860b865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:33:59.710608 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695403d9-1713-49d3-bee8-39c47860b865 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:33:59.710890 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695403d9-1713-49d3-bee8-39c47860b865 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:33:59.711164 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695403d9-1713-49d3-bee8-39c47860b865 None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:33:59.711875 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695403d9-1713-49d3-bee8-39c47860b865 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:33:59.712209 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695403d9-1713-49d3-bee8-39c47860b865 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:33:59.713599 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695403d9-1713-49d3-bee8-39c47860b865 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:33:59.726113 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 636/1257] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jun 16 17:33:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:33:59.731759 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb605e7-64c9-424f-859e-9a2d9ee63ae0 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:33:59.830655 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb605e7-64c9-424f-859e-9a2d9ee63ae0 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:33:59.864807 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc4c5e2-6195-48e4-b4bb-dcb21ca5ea17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:33:59.865601 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc4c5e2-6195-48e4-b4bb-dcb21ca5ea17 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:33:59.865881 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc4c5e2-6195-48e4-b4bb-dcb21ca5ea17 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:33:59.866157 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc4c5e2-6195-48e4-b4bb-dcb21ca5ea17 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:33:59.873386 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc4c5e2-6195-48e4-b4bb-dcb21ca5ea17 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:33:59.875398 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc4c5e2-6195-48e4-b4bb-dcb21ca5ea17 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:33:59.889916 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-dfc4c5e2-6195-48e4-b4bb-dcb21ca5ea17 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:33:59.889916 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:33:59.889916 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:33:59.889916 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:33:59.889916 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:33:59.889916 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:33:59.889916 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:33:59.889916 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:33:59.889916 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:33:59.889916 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:33:59.889916 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:33:59.889916 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:33:59.889916 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:33:59.889916 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:33:59.889916 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:33:59.891351 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:33:59.891351 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:33:59.891351 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:33:59.891351 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:33:59.891351 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:33:59.891351 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:33:59.891351 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:33:59.891351 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:33:59.891351 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:33:59.891351 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:33:59.891351 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:33:59.891351 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:33:59.891351 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:33:59.891351 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:33:59.891351 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:33:59.891351 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:33:59.891351 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:33:59.892252 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:33:59.892252 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:33:59.892252 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:33:59.892252 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:33:59.892252 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:33:59.892252 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:33:59.892252 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:33:59.892252 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:33:59.892252 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 622/1258] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jun 16 17:33:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:33:59.897308 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695403d9-1713-49d3-bee8-39c47860b865 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:33:59.992951 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695403d9-1713-49d3-bee8-39c47860b865 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:00.026005 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba2071c-7e89-406b-b405-3e10c36329b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:00.026653 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba2071c-7e89-406b-b405-3e10c36329b7 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:00.026934 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba2071c-7e89-406b-b405-3e10c36329b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:00.027209 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba2071c-7e89-406b-b405-3e10c36329b7 None admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:00.027973 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba2071c-7e89-406b-b405-3e10c36329b7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:34:00.028277 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba2071c-7e89-406b-b405-3e10c36329b7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:00.029948 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba2071c-7e89-406b-b405-3e10c36329b7 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:00.041319 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 637/1259] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Jun 16 17:33:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:34:00.047465 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc4c5e2-6195-48e4-b4bb-dcb21ca5ea17 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:00.072982 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:34:00.163953 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc4c5e2-6195-48e4-b4bb-dcb21ca5ea17 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:00.198068 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450b75bd-f026-4ba5-b766-a4f091723615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:00.198713 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450b75bd-f026-4ba5-b766-a4f091723615 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:00.198994 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450b75bd-f026-4ba5-b766-a4f091723615 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:00.199272 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450b75bd-f026-4ba5-b766-a4f091723615 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:00.200081 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450b75bd-f026-4ba5-b766-a4f091723615 None admin] RBAC: Adding query filter params (role.id=3ca2dcbbd29d4a31a3ab78f2299bc239, scope.project.id=8abb87f2f141451a8c6ed832db82be18, user.id=0570d8d9508d4546ae897276f7d181d2) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:34:00.200407 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450b75bd-f026-4ba5-b766-a4f091723615 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:00.201870 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450b75bd-f026-4ba5-b766-a4f091723615 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:00.207172 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 623/1260] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jun 16 17:34:00 2022] GET /identity/v3/role_assignments?user.id=0570d8d9508d4546ae897276f7d181d2&scope.project.id=8abb87f2f141451a8c6ed832db82be18&role.id=3ca2dcbbd29d4a31a3ab78f2299bc239 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:34:02.370912 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a255361-04ba-4f0c-9eaa-c27aad6e9a6c None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:02.371462 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a255361-04ba-4f0c-9eaa-c27aad6e9a6c None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:02.371462 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a255361-04ba-4f0c-9eaa-c27aad6e9a6c None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:02.373796 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 638/1261] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:34:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:34:02.379863 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d078ec57-0b4f-4865-9e6f-40dfc3e12b79 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:02.380323 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d078ec57-0b4f-4865-9e6f-40dfc3e12b79 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:02.380547 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d078ec57-0b4f-4865-9e6f-40dfc3e12b79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:02.400267 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-d078ec57-0b4f-4865-9e6f-40dfc3e12b79 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:34:02.401519 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-d078ec57-0b4f-4865-9e6f-40dfc3e12b79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:02.479078 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 624/1262] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:34:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:34:02.502111 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a255361-04ba-4f0c-9eaa-c27aad6e9a6c None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:02.512783 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:34:02.621805 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a255361-04ba-4f0c-9eaa-c27aad6e9a6c None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:02.654531 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca117936-f841-4799-a9c4-55642c371b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:02.655306 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca117936-f841-4799-a9c4-55642c371b77 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:02.655586 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca117936-f841-4799-a9c4-55642c371b77 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:02.655863 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca117936-f841-4799-a9c4-55642c371b77 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:02.665509 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca117936-f841-4799-a9c4-55642c371b77 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:34:02.673551 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-ca117936-f841-4799-a9c4-55642c371b77 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:34:02.673551 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:34:02.673551 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:34:02.673551 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:34:02.673551 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:34:02.673551 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:34:02.673551 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:34:02.673551 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:34:02.673551 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:34:02.673551 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:34:02.673551 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:34:02.673551 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:34:02.673551 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:34:02.673551 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:34:02.673551 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:34:02.673551 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:34:02.674636 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:34:02.674636 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:34:02.674636 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:34:02.674636 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:34:02.674636 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:34:02.674636 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:34:02.674636 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:34:02.674636 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:34:02.674636 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:34:02.674636 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:34:02.674636 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:34:02.674636 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:34:02.674636 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:34:02.674636 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:34:02.674636 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:34:02.674636 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:34:02.675845 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:34:02.675845 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:34:02.675845 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:34:02.675845 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:34:02.675845 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:34:02.675845 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:34:02.675845 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:34:02.675845 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:34:02.675845 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:34:02.675845 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:34:02.675845 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:34:02.675845 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:34:02.675845 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 639/1263] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Jun 16 17:34:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:34:02.680952 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d078ec57-0b4f-4865-9e6f-40dfc3e12b79 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:02.699281 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:34:02.779670 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d078ec57-0b4f-4865-9e6f-40dfc3e12b79 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:02.813560 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccb5d08-a6b3-4bd2-9bd3-acb4d5837775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:02.814220 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccb5d08-a6b3-4bd2-9bd3-acb4d5837775 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:02.814512 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccb5d08-a6b3-4bd2-9bd3-acb4d5837775 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:02.814788 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccb5d08-a6b3-4bd2-9bd3-acb4d5837775 None admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:02.815528 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccb5d08-a6b3-4bd2-9bd3-acb4d5837775 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:34:02.815835 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccb5d08-a6b3-4bd2-9bd3-acb4d5837775 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:02.817252 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccb5d08-a6b3-4bd2-9bd3-acb4d5837775 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:02.825149 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 625/1264] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jun 16 17:34:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:34:02.831872 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca117936-f841-4799-a9c4-55642c371b77 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:02.898773 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:34:02.932867 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca117936-f841-4799-a9c4-55642c371b77 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:02.965725 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55200033-abca-48db-bb06-2f964b161a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:02.966620 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55200033-abca-48db-bb06-2f964b161a79 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:02.966904 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55200033-abca-48db-bb06-2f964b161a79 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:02.967185 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55200033-abca-48db-bb06-2f964b161a79 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:02.977289 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55200033-abca-48db-bb06-2f964b161a79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:02.978713 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55200033-abca-48db-bb06-2f964b161a79 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:02.995064 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-55200033-abca-48db-bb06-2f964b161a79 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:34:02.995064 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:34:02.995064 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:34:02.995064 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:34:02.995064 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:34:02.995064 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:34:02.995064 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:34:02.995064 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:34:02.995064 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:34:02.995064 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:34:02.995064 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:34:02.995064 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:34:02.995064 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:34:02.995064 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:34:02.995064 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:34:02.995958 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:34:02.995958 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:34:02.995958 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:34:02.995958 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:34:02.995958 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:34:02.995958 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:34:02.995958 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:34:02.995958 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:34:02.995958 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:34:02.995958 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:34:02.995958 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:34:02.995958 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:34:02.995958 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:34:02.995958 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:34:02.995958 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:34:02.995958 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:34:02.995958 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:34:02.996938 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:34:02.996938 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:34:02.996938 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:34:02.996938 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:34:02.996938 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 640/1265] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jun 16 17:34:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:34:03.002628 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccb5d08-a6b3-4bd2-9bd3-acb4d5837775 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:03.086494 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:34:03.105413 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccb5d08-a6b3-4bd2-9bd3-acb4d5837775 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:03.139953 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd40cf5-9794-4ef0-86b5-3968b0709f55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:03.140697 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd40cf5-9794-4ef0-86b5-3968b0709f55 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:03.140988 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd40cf5-9794-4ef0-86b5-3968b0709f55 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:03.141267 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd40cf5-9794-4ef0-86b5-3968b0709f55 None admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:03.141992 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd40cf5-9794-4ef0-86b5-3968b0709f55 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:34:03.142300 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd40cf5-9794-4ef0-86b5-3968b0709f55 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:03.143664 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd40cf5-9794-4ef0-86b5-3968b0709f55 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:03.155955 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 626/1266] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jun 16 17:34:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:34:03.161650 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55200033-abca-48db-bb06-2f964b161a79 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:03.274545 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55200033-abca-48db-bb06-2f964b161a79 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:03.294124 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:34:03.309041 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8700df54-44bc-40fd-91af-e0edc8524ed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:03.309840 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8700df54-44bc-40fd-91af-e0edc8524ed3 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:03.310128 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8700df54-44bc-40fd-91af-e0edc8524ed3 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:03.311034 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8700df54-44bc-40fd-91af-e0edc8524ed3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:03.318338 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8700df54-44bc-40fd-91af-e0edc8524ed3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:03.319771 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8700df54-44bc-40fd-91af-e0edc8524ed3 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:03.334069 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-8700df54-44bc-40fd-91af-e0edc8524ed3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 16 17:34:03.334069 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:34:03.334069 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:34:03.334069 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:34:03.334069 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:34:03.334069 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:34:03.334069 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:34:03.334069 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:34:03.334069 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:34:03.334069 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:34:03.334069 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:34:03.334069 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:34:03.334069 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:34:03.334069 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:34:03.334069 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:34:03.334069 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:34:03.335181 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:34:03.335181 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:34:03.335181 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:34:03.335181 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:34:03.335181 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:34:03.335181 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:34:03.335181 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:34:03.335181 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:34:03.335181 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:34:03.335181 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:34:03.335181 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:34:03.335181 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:34:03.335181 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:34:03.335181 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:34:03.335181 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:34:03.335181 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:34:03.335181 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:34:03.335181 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:34:03.336363 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:34:03.336363 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:34:03.336363 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:34:03.336363 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:34:03.336363 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:34:03.336363 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:34:03.336363 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:34:03.336363 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:34:03.336363 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 16 17:34:03.336363 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:34:03.336363 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 641/1267] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jun 16 17:34:03 2022] GET /identity/v3/users/placement => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:34:03.341096 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd40cf5-9794-4ef0-86b5-3968b0709f55 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:03.451683 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd40cf5-9794-4ef0-86b5-3968b0709f55 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:03.484925 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c784f773-bf01-4e3f-bc50-5e13ce9a6356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:03.485593 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c784f773-bf01-4e3f-bc50-5e13ce9a6356 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:03.485879 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c784f773-bf01-4e3f-bc50-5e13ce9a6356 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:03.486154 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c784f773-bf01-4e3f-bc50-5e13ce9a6356 None admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:03.486986 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c784f773-bf01-4e3f-bc50-5e13ce9a6356 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:34:03.487297 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c784f773-bf01-4e3f-bc50-5e13ce9a6356 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:03.488744 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c784f773-bf01-4e3f-bc50-5e13ce9a6356 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:03.497152 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:34:03.506476 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 627/1268] 10.222.0.42 () {56 vars in 1143 bytes} [Thu Jun 16 17:34:03 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:34:03.512509 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8700df54-44bc-40fd-91af-e0edc8524ed3 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:03.610873 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8700df54-44bc-40fd-91af-e0edc8524ed3 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:03.644079 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fe4059-2f5d-4c71-a2ea-7d67882ccd18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:03.645209 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fe4059-2f5d-4c71-a2ea-7d67882ccd18 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:03.645495 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fe4059-2f5d-4c71-a2ea-7d67882ccd18 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:03.645775 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fe4059-2f5d-4c71-a2ea-7d67882ccd18 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:03.656211 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fe4059-2f5d-4c71-a2ea-7d67882ccd18 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:03.657797 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fe4059-2f5d-4c71-a2ea-7d67882ccd18 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:03.674764 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-01fe4059-2f5d-4c71-a2ea-7d67882ccd18 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:34:03.674764 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:34:03.674764 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:34:03.674764 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:34:03.674764 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:34:03.674764 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:34:03.674764 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:34:03.674764 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:34:03.674764 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:34:03.674764 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:34:03.674764 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:34:03.674764 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:34:03.674764 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:34:03.674764 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:34:03.674764 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:34:03.675833 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:34:03.675833 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:34:03.675833 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:34:03.675833 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:34:03.675833 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:34:03.675833 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:34:03.675833 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:34:03.675833 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:34:03.675833 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:34:03.675833 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:34:03.675833 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:34:03.675833 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:34:03.675833 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:34:03.675833 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:34:03.675833 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:34:03.675833 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:34:03.675833 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:34:03.677001 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:34:03.677001 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:34:03.677001 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:34:03.677001 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:34:03.677001 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 642/1269] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jun 16 17:34:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:34:03.681574 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c784f773-bf01-4e3f-bc50-5e13ce9a6356 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:03.785215 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c784f773-bf01-4e3f-bc50-5e13ce9a6356 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:03.818845 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbdebc2-7873-4760-a150-8dd01ff28881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:03.819498 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbdebc2-7873-4760-a150-8dd01ff28881 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:03.819784 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbdebc2-7873-4760-a150-8dd01ff28881 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:03.820059 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbdebc2-7873-4760-a150-8dd01ff28881 None admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:03.820873 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbdebc2-7873-4760-a150-8dd01ff28881 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:34:03.821214 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbdebc2-7873-4760-a150-8dd01ff28881 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:03.822577 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbdebc2-7873-4760-a150-8dd01ff28881 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:03.835000 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 628/1270] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jun 16 17:34:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:34:03.840813 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fe4059-2f5d-4c71-a2ea-7d67882ccd18 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:03.872241 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:34:03.943130 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fe4059-2f5d-4c71-a2ea-7d67882ccd18 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:03.974895 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a8a896-bfc3-41ae-a0ea-b08f21956a24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:03.975677 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a8a896-bfc3-41ae-a0ea-b08f21956a24 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:03.975980 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a8a896-bfc3-41ae-a0ea-b08f21956a24 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:03.976241 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a8a896-bfc3-41ae-a0ea-b08f21956a24 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:03.983227 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a8a896-bfc3-41ae-a0ea-b08f21956a24 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:03.984684 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a8a896-bfc3-41ae-a0ea-b08f21956a24 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:03.999293 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-34a8a896-bfc3-41ae-a0ea-b08f21956a24 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:34:03.999293 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:34:03.999293 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:34:03.999293 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:34:03.999293 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:34:03.999293 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:34:03.999293 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:34:03.999293 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:34:03.999293 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:34:03.999293 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:34:03.999293 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:34:03.999293 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:34:03.999293 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:34:03.999293 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:34:03.999293 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:34:04.000173 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:34:04.000173 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:34:04.000173 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:34:04.000173 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:34:04.000173 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:34:04.000173 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:34:04.000173 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:34:04.000173 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:34:04.000173 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:34:04.000173 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:34:04.000173 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:34:04.000173 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:34:04.000173 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:34:04.000173 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:34:04.000173 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:34:04.000173 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:34:04.000173 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:34:04.001302 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:34:04.001302 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:34:04.001302 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:34:04.001302 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:34:04.001302 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:34:04.001302 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:34:04.001302 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:34:04.001302 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:34:04.001302 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 643/1271] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jun 16 17:34:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:34:04.006319 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbdebc2-7873-4760-a150-8dd01ff28881 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:04.065240 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:34:04.105917 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbdebc2-7873-4760-a150-8dd01ff28881 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:04.140024 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266e1691-cf38-48bf-80df-3c464ad11b52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:04.140767 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266e1691-cf38-48bf-80df-3c464ad11b52 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:04.141051 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266e1691-cf38-48bf-80df-3c464ad11b52 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:04.141324 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266e1691-cf38-48bf-80df-3c464ad11b52 None admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:04.142100 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266e1691-cf38-48bf-80df-3c464ad11b52 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:34:04.142405 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266e1691-cf38-48bf-80df-3c464ad11b52 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:04.143825 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266e1691-cf38-48bf-80df-3c464ad11b52 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:04.156196 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 629/1272] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Jun 16 17:34:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:34:04.162162 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a8a896-bfc3-41ae-a0ea-b08f21956a24 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:04.255942 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:34:04.264765 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a8a896-bfc3-41ae-a0ea-b08f21956a24 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:04.305177 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df17add-c9a0-4271-9a23-b861b0e9daaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:04.306062 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df17add-c9a0-4271-9a23-b861b0e9daaf None admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:04.306345 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df17add-c9a0-4271-9a23-b861b0e9daaf None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:04.306624 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df17add-c9a0-4271-9a23-b861b0e9daaf None admin] PATH_INFO: `/v3/projects/8abb87f2f141451a8c6ed832db82be18/users/0570d8d9508d4546ae897276f7d181d2/roles/3ca2dcbbd29d4a31a3ab78f2299bc239` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:04.308600 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df17add-c9a0-4271-9a23-b861b0e9daaf None admin] RBAC: Authorizing `identity:create_grant(project_id=8abb87f2f141451a8c6ed832db82be18, user_id=0570d8d9508d4546ae897276f7d181d2, role_id=3ca2dcbbd29d4a31a3ab78f2299bc239)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:04.310037 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df17add-c9a0-4271-9a23-b861b0e9daaf None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:04.323132 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 644/1273] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Jun 16 17:34:04 2022] PUT /identity/v3/projects/8abb87f2f141451a8c6ed832db82be18/users/0570d8d9508d4546ae897276f7d181d2/roles/3ca2dcbbd29d4a31a3ab78f2299bc239 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:34:06.478846 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3a600b-9e36-4768-8215-333bba9d176f None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:06.479405 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3a600b-9e36-4768-8215-333bba9d176f None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:06.479405 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3a600b-9e36-4768-8215-333bba9d176f None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:06.480829 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 630/1274] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:34:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:34:06.486347 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318f3861-9d78-48b0-b8d8-c7173cebdbdb None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:06.486766 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318f3861-9d78-48b0-b8d8-c7173cebdbdb None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:06.486894 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318f3861-9d78-48b0-b8d8-c7173cebdbdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:06.503957 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-318f3861-9d78-48b0-b8d8-c7173cebdbdb None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:34:06.505232 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-318f3861-9d78-48b0-b8d8-c7173cebdbdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:06.570207 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 645/1275] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:34:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:34:06.594551 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3a600b-9e36-4768-8215-333bba9d176f None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:06.695837 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3a600b-9e36-4768-8215-333bba9d176f None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:06.712575 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:34:06.730120 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a8d3ca-ede9-4322-9fba-14a437cbdbb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:06.730840 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a8d3ca-ede9-4322-9fba-14a437cbdbb3 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:06.731118 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a8d3ca-ede9-4322-9fba-14a437cbdbb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:06.731387 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a8d3ca-ede9-4322-9fba-14a437cbdbb3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:06.741908 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a8d3ca-ede9-4322-9fba-14a437cbdbb3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:34:06.749520 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-28a8d3ca-ede9-4322-9fba-14a437cbdbb3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:34:06.749520 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:34:06.749520 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:34:06.749520 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:34:06.749520 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:34:06.749520 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:34:06.749520 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:34:06.749520 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:34:06.749520 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:34:06.749520 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:34:06.749520 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:34:06.749520 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:34:06.749520 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:34:06.749520 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:34:06.749520 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:34:06.749520 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:34:06.750725 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:34:06.750725 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:34:06.750725 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:34:06.750725 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:34:06.750725 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:34:06.750725 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:34:06.750725 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:34:06.750725 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:34:06.750725 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:34:06.750725 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:34:06.750725 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:34:06.750725 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:34:06.750725 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:34:06.750725 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:34:06.750725 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:34:06.750725 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:34:06.751785 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:34:06.751785 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:34:06.751785 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:34:06.751785 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:34:06.751785 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:34:06.751785 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:34:06.751785 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:34:06.751785 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:34:06.751785 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:34:06.751785 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:34:06.751785 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:34:06.751785 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:34:06.751785 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 631/1276] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Jun 16 17:34:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:34:06.756735 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318f3861-9d78-48b0-b8d8-c7173cebdbdb None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:06.836736 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:34:06.855538 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318f3861-9d78-48b0-b8d8-c7173cebdbdb None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:06.891215 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d50c8ac-b3a0-4110-a917-2eeaca36bf39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:06.891880 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d50c8ac-b3a0-4110-a917-2eeaca36bf39 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:06.892163 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d50c8ac-b3a0-4110-a917-2eeaca36bf39 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:06.892467 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d50c8ac-b3a0-4110-a917-2eeaca36bf39 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:06.893206 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d50c8ac-b3a0-4110-a917-2eeaca36bf39 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:34:06.893512 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d50c8ac-b3a0-4110-a917-2eeaca36bf39 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:06.895116 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d50c8ac-b3a0-4110-a917-2eeaca36bf39 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:06.903095 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 646/1277] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jun 16 17:34:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:34:06.909398 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a8d3ca-ede9-4322-9fba-14a437cbdbb3 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:07.006731 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a8d3ca-ede9-4322-9fba-14a437cbdbb3 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:07.040277 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c76de57-e38e-4bb9-9b93-41c3cb09cd96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:07.041087 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c76de57-e38e-4bb9-9b93-41c3cb09cd96 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:07.041367 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c76de57-e38e-4bb9-9b93-41c3cb09cd96 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:07.041641 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c76de57-e38e-4bb9-9b93-41c3cb09cd96 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:07.052242 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c76de57-e38e-4bb9-9b93-41c3cb09cd96 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:07.053723 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c76de57-e38e-4bb9-9b93-41c3cb09cd96 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:07.070464 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-2c76de57-e38e-4bb9-9b93-41c3cb09cd96 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:34:07.070464 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:34:07.070464 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:34:07.070464 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:34:07.070464 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:34:07.070464 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:34:07.070464 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:34:07.070464 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:34:07.070464 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:34:07.070464 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:34:07.070464 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:34:07.070464 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:34:07.070464 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:34:07.070464 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:34:07.070464 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:34:07.071541 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:34:07.071541 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:34:07.071541 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:34:07.071541 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:34:07.071541 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:34:07.071541 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:34:07.071541 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:34:07.071541 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:34:07.071541 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:34:07.071541 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:34:07.071541 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:34:07.071541 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:34:07.071541 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:34:07.071541 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:34:07.071541 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:34:07.071541 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:34:07.071541 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:34:07.072731 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:34:07.072731 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:34:07.072731 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:34:07.072731 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:34:07.072731 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 632/1278] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jun 16 17:34:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:34:07.078019 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d50c8ac-b3a0-4110-a917-2eeaca36bf39 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:07.182785 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d50c8ac-b3a0-4110-a917-2eeaca36bf39 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:07.216106 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c79e8c-b1de-4577-9274-a110eebbc9e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:07.216852 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c79e8c-b1de-4577-9274-a110eebbc9e7 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:07.217138 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c79e8c-b1de-4577-9274-a110eebbc9e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:07.217418 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c79e8c-b1de-4577-9274-a110eebbc9e7 None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:07.218131 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c79e8c-b1de-4577-9274-a110eebbc9e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:34:07.218439 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c79e8c-b1de-4577-9274-a110eebbc9e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:07.219810 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c79e8c-b1de-4577-9274-a110eebbc9e7 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:07.231208 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 647/1279] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jun 16 17:34:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:34:07.237392 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c76de57-e38e-4bb9-9b93-41c3cb09cd96 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:07.269068 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:34:07.337634 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c76de57-e38e-4bb9-9b93-41c3cb09cd96 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:07.375923 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39253a0b-0a64-4564-bd1c-ca956520f8b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:07.376740 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39253a0b-0a64-4564-bd1c-ca956520f8b8 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:07.377004 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39253a0b-0a64-4564-bd1c-ca956520f8b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:07.377308 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39253a0b-0a64-4564-bd1c-ca956520f8b8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:07.385236 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39253a0b-0a64-4564-bd1c-ca956520f8b8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:07.386677 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39253a0b-0a64-4564-bd1c-ca956520f8b8 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:07.402494 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-39253a0b-0a64-4564-bd1c-ca956520f8b8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 16 17:34:07.402494 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:34:07.402494 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:34:07.402494 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:34:07.402494 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:34:07.402494 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:34:07.402494 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:34:07.402494 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:34:07.402494 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:34:07.402494 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:34:07.402494 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:34:07.402494 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:34:07.402494 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:34:07.402494 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:34:07.402494 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:34:07.402494 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:34:07.403497 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:34:07.403497 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:34:07.403497 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:34:07.403497 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:34:07.403497 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:34:07.403497 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:34:07.403497 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:34:07.403497 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:34:07.403497 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:34:07.403497 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:34:07.403497 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:34:07.403497 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:34:07.403497 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:34:07.403497 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:34:07.403497 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:34:07.403497 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:34:07.403497 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:34:07.403497 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:34:07.405123 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:34:07.405123 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:34:07.405123 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:34:07.405123 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:34:07.405123 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:34:07.405123 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:34:07.405123 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:34:07.405123 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:34:07.405123 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 16 17:34:07.405123 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:34:07.405123 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 633/1280] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jun 16 17:34:07 2022] GET /identity/v3/users/placement => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:34:07.410087 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c79e8c-b1de-4577-9274-a110eebbc9e7 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:07.414394 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:34:07.514935 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c79e8c-b1de-4577-9274-a110eebbc9e7 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:07.548542 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156daa34-cc8e-42ae-8111-2bfe27851167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:07.549141 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156daa34-cc8e-42ae-8111-2bfe27851167 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:07.549422 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156daa34-cc8e-42ae-8111-2bfe27851167 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:07.549696 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156daa34-cc8e-42ae-8111-2bfe27851167 None admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:07.550512 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156daa34-cc8e-42ae-8111-2bfe27851167 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:34:07.550817 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156daa34-cc8e-42ae-8111-2bfe27851167 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:07.552226 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156daa34-cc8e-42ae-8111-2bfe27851167 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:07.568820 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 648/1281] 10.222.0.42 () {56 vars in 1143 bytes} [Thu Jun 16 17:34:07 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:34:07.574671 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39253a0b-0a64-4564-bd1c-ca956520f8b8 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:07.663534 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:34:07.673370 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39253a0b-0a64-4564-bd1c-ca956520f8b8 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:07.706288 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cb984f-6e28-4db1-8f48-9646003e65d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:07.706929 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cb984f-6e28-4db1-8f48-9646003e65d4 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:07.707210 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cb984f-6e28-4db1-8f48-9646003e65d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:07.707481 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cb984f-6e28-4db1-8f48-9646003e65d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:07.774981 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cb984f-6e28-4db1-8f48-9646003e65d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:07.776473 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cb984f-6e28-4db1-8f48-9646003e65d4 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:07.862911 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:34:07.873522 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 634/1282] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jun 16 17:34:07 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 17:34:07.880699 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156daa34-cc8e-42ae-8111-2bfe27851167 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:07.943447 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:34:07.981310 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156daa34-cc8e-42ae-8111-2bfe27851167 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:08.015850 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64348d9-fa56-41a5-8625-37a54e0dacea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:08.016655 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64348d9-fa56-41a5-8625-37a54e0dacea None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:08.016936 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64348d9-fa56-41a5-8625-37a54e0dacea None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:08.017209 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64348d9-fa56-41a5-8625-37a54e0dacea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:08.027168 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64348d9-fa56-41a5-8625-37a54e0dacea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:08.028624 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64348d9-fa56-41a5-8625-37a54e0dacea None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:08.050508 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-c64348d9-fa56-41a5-8625-37a54e0dacea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:34:08.050508 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:34:08.050508 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:34:08.050508 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:34:08.050508 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:34:08.050508 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:34:08.050508 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:34:08.050508 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:34:08.050508 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:34:08.050508 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:34:08.050508 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:34:08.050508 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:34:08.050508 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:34:08.050508 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:34:08.050508 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:34:08.051602 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:34:08.051602 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:34:08.051602 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:34:08.051602 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:34:08.051602 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:34:08.051602 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:34:08.051602 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:34:08.051602 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:34:08.051602 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:34:08.051602 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:34:08.051602 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:34:08.051602 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:34:08.051602 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:34:08.051602 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:34:08.051602 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:34:08.051602 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:34:08.051602 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:34:08.052786 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:34:08.052786 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:34:08.052786 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:34:08.052786 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:34:08.052786 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 649/1283] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jun 16 17:34:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:34:08.057532 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cb984f-6e28-4db1-8f48-9646003e65d4 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:08.153480 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cb984f-6e28-4db1-8f48-9646003e65d4 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:08.186826 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bed296-c19e-4817-8b9e-02a0faad3368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:08.187469 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bed296-c19e-4817-8b9e-02a0faad3368 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:08.187753 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bed296-c19e-4817-8b9e-02a0faad3368 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:08.188030 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bed296-c19e-4817-8b9e-02a0faad3368 None admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:08.188769 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bed296-c19e-4817-8b9e-02a0faad3368 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:34:08.189076 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bed296-c19e-4817-8b9e-02a0faad3368 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:08.190444 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bed296-c19e-4817-8b9e-02a0faad3368 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:08.201374 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 635/1284] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jun 16 17:34:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:34:08.207109 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64348d9-fa56-41a5-8625-37a54e0dacea None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:08.303846 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64348d9-fa56-41a5-8625-37a54e0dacea None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:08.317432 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:34:08.337353 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117cd7fe-17d6-4a5a-9bd0-51ce3bbeb7ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:08.338140 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117cd7fe-17d6-4a5a-9bd0-51ce3bbeb7ef None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:08.338422 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117cd7fe-17d6-4a5a-9bd0-51ce3bbeb7ef None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:08.338696 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117cd7fe-17d6-4a5a-9bd0-51ce3bbeb7ef None admin] PATH_INFO: `/v3/projects/service` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:08.346227 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117cd7fe-17d6-4a5a-9bd0-51ce3bbeb7ef None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:08.347657 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117cd7fe-17d6-4a5a-9bd0-51ce3bbeb7ef None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:08.365091 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-117cd7fe-17d6-4a5a-9bd0-51ce3bbeb7ef None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:34:08.365091 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:34:08.365091 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:34:08.365091 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:34:08.365091 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:34:08.365091 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:34:08.365091 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:34:08.365091 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:34:08.365091 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:34:08.365091 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:34:08.365091 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:34:08.365091 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:34:08.365091 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:34:08.365091 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:34:08.365091 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:34:08.366147 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:34:08.366147 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:34:08.366147 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:34:08.366147 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:34:08.366147 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:34:08.366147 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:34:08.366147 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:34:08.366147 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:34:08.366147 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Jun 16 17:34:08.366147 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:34:08.366147 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:34:08.366147 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:34:08.366147 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:34:08.366147 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:34:08.366147 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:34:08.366147 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:34:08.366147 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:34:08.367069 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:34:08.367069 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:34:08.367069 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:34:08.367069 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:34:08.367069 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:34:08.367069 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:34:08.367069 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:34:08.367069 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Jun 16 17:34:08.367069 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 650/1285] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jun 16 17:34:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:34:08.372538 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bed296-c19e-4817-8b9e-02a0faad3368 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:08.423099 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:34:08.479071 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bed296-c19e-4817-8b9e-02a0faad3368 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:08.511971 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fb2fa0-0284-407c-a998-070f409e1d05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:08.512657 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fb2fa0-0284-407c-a998-070f409e1d05 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:08.512944 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fb2fa0-0284-407c-a998-070f409e1d05 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:08.513221 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fb2fa0-0284-407c-a998-070f409e1d05 None admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:08.514538 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fb2fa0-0284-407c-a998-070f409e1d05 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:34:08.514842 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fb2fa0-0284-407c-a998-070f409e1d05 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:08.516263 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fb2fa0-0284-407c-a998-070f409e1d05 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:08.528340 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 636/1286] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Jun 16 17:34:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:34:08.535087 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117cd7fe-17d6-4a5a-9bd0-51ce3bbeb7ef None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:08.630254 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117cd7fe-17d6-4a5a-9bd0-51ce3bbeb7ef None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:08.662970 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f266884a-ae57-42ca-8d3c-68e83968e396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:08.663598 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f266884a-ae57-42ca-8d3c-68e83968e396 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:08.663876 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f266884a-ae57-42ca-8d3c-68e83968e396 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:08.664153 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f266884a-ae57-42ca-8d3c-68e83968e396 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:08.664991 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f266884a-ae57-42ca-8d3c-68e83968e396 None admin] RBAC: Adding query filter params (role.id=3ca2dcbbd29d4a31a3ab78f2299bc239, scope.project.id=8abb87f2f141451a8c6ed832db82be18, user.id=0570d8d9508d4546ae897276f7d181d2) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:34:08.665298 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f266884a-ae57-42ca-8d3c-68e83968e396 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:08.666699 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f266884a-ae57-42ca-8d3c-68e83968e396 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:08.672476 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 651/1287] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jun 16 17:34:08 2022] GET /identity/v3/role_assignments?user.id=0570d8d9508d4546ae897276f7d181d2&scope.project.id=8abb87f2f141451a8c6ed832db82be18&role.id=3ca2dcbbd29d4a31a3ab78f2299bc239 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:34:10.837465 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec3367d-79f9-480c-a496-57082eacb2d9 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:10.838000 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec3367d-79f9-480c-a496-57082eacb2d9 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:10.838000 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec3367d-79f9-480c-a496-57082eacb2d9 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:10.839392 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 637/1288] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:34:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:34:10.845283 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa67c989-f7f1-4e2f-9899-b19ef415cfd6 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:10.845666 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa67c989-f7f1-4e2f-9899-b19ef415cfd6 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:10.845845 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa67c989-f7f1-4e2f-9899-b19ef415cfd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:10.862575 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-fa67c989-f7f1-4e2f-9899-b19ef415cfd6 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:34:10.863833 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-fa67c989-f7f1-4e2f-9899-b19ef415cfd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:10.894170 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:34:10.934314 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 652/1289] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:34:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:34:10.958001 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec3367d-79f9-480c-a496-57082eacb2d9 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:11.062625 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec3367d-79f9-480c-a496-57082eacb2d9 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:11.072817 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:34:11.098458 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f11ba1-462f-4e43-ba4e-c6c7112ec0e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:11.099431 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f11ba1-462f-4e43-ba4e-c6c7112ec0e1 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:11.099716 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f11ba1-462f-4e43-ba4e-c6c7112ec0e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:11.099996 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f11ba1-462f-4e43-ba4e-c6c7112ec0e1 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:11.100696 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f11ba1-462f-4e43-ba4e-c6c7112ec0e1 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:11.102066 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f11ba1-462f-4e43-ba4e-c6c7112ec0e1 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:11.114383 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-b9f11ba1-462f-4e43-ba4e-c6c7112ec0e1 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 17:34:11.114383 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:34:11.114383 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 17:34:11.114383 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 17:34:11.114383 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 17:34:11.114383 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 17:34:11.114383 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 17:34:11.114383 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:34:11.114383 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 17:34:11.114383 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:34:11.114383 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:34:11.114383 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:34:11.114383 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:34:11.114383 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:34:11.114383 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:34:11.114383 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:34:11.114383 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:34:11.115614 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:34:11.115614 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:34:11.115614 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:34:11.115614 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:34:11.115614 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:34:11.115614 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:34:11.115614 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 17:34:11.115614 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 17:34:11.115614 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 17:34:11.115614 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 17:34:11.115614 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:34:11.115614 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:34:11.115614 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:34:11.115614 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:34:11.115614 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:34:11.115614 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:34:11.116723 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:34:11.116723 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:34:11.116723 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:34:11.116723 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:34:11.116723 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:34:11.116723 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:34:11.116723 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:34:11.116723 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:34:11.116723 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:34:11.116723 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:34:11.116723 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 17:34:11.116723 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:34:11.116723 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 17:34:11.116723 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:34:11.116723 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 638/1290] 10.222.0.42 () {56 vars in 1117 bytes} [Thu Jun 16 17:34:10 2022] GET /identity/v3/services/placement => generated 90 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:34:11.121654 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa67c989-f7f1-4e2f-9899-b19ef415cfd6 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:11.218000 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa67c989-f7f1-4e2f-9899-b19ef415cfd6 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:11.251496 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee93f3b1-d1e6-49e6-8289-26d8c9243291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:11.252145 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee93f3b1-d1e6-49e6-8289-26d8c9243291 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:11.252456 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee93f3b1-d1e6-49e6-8289-26d8c9243291 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:11.252733 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee93f3b1-d1e6-49e6-8289-26d8c9243291 None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:11.253424 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee93f3b1-d1e6-49e6-8289-26d8c9243291 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:34:11.253731 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee93f3b1-d1e6-49e6-8289-26d8c9243291 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:11.255081 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee93f3b1-d1e6-49e6-8289-26d8c9243291 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:11.259772 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 653/1291] 10.222.0.42 () {56 vars in 1116 bytes} [Thu Jun 16 17:34:11 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:34:11.265825 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f11ba1-462f-4e43-ba4e-c6c7112ec0e1 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:11.367916 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f11ba1-462f-4e43-ba4e-c6c7112ec0e1 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:11.407610 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9354a8bd-c53d-4f83-a112-93fdb905fa48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:11.408277 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9354a8bd-c53d-4f83-a112-93fdb905fa48 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:11.408593 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9354a8bd-c53d-4f83-a112-93fdb905fa48 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:11.408882 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9354a8bd-c53d-4f83-a112-93fdb905fa48 None admin] PATH_INFO: `/v3/services` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:11.409577 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9354a8bd-c53d-4f83-a112-93fdb905fa48 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:34:11.409886 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9354a8bd-c53d-4f83-a112-93fdb905fa48 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:11.411248 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9354a8bd-c53d-4f83-a112-93fdb905fa48 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:11.415933 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 639/1292] 10.222.0.42 () {56 vars in 1116 bytes} [Thu Jun 16 17:34:11 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:34:13.522990 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27943e4f-9e3d-4584-89f2-ca0a1829751c None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:13.523551 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27943e4f-9e3d-4584-89f2-ca0a1829751c None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:13.523551 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27943e4f-9e3d-4584-89f2-ca0a1829751c None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:13.524940 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 654/1293] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:34:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:34:13.530695 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb305f22-0e40-46f4-b984-966d7fa6936c None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:13.531095 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb305f22-0e40-46f4-b984-966d7fa6936c None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:13.531246 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb305f22-0e40-46f4-b984-966d7fa6936c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:13.556780 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-cb305f22-0e40-46f4-b984-966d7fa6936c None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:34:13.557976 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-cb305f22-0e40-46f4-b984-966d7fa6936c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:13.594698 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:34:13.631890 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 640/1294] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:34:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:34:13.656031 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27943e4f-9e3d-4584-89f2-ca0a1829751c None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:13.686557 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:34:13.756666 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27943e4f-9e3d-4584-89f2-ca0a1829751c None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:13.791289 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089ebe1a-dffd-4531-88f6-0861bac3625c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:13.791876 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089ebe1a-dffd-4531-88f6-0861bac3625c None admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:13.792157 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089ebe1a-dffd-4531-88f6-0861bac3625c None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:13.792453 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089ebe1a-dffd-4531-88f6-0861bac3625c None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:13.793162 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089ebe1a-dffd-4531-88f6-0861bac3625c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:13.794512 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089ebe1a-dffd-4531-88f6-0861bac3625c None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:13.808732 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 655/1295] 10.222.0.42 () {60 vars in 1141 bytes} [Thu Jun 16 17:34:13 2022] POST /identity/v3/services => generated 246 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:34:15.952644 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5384c5c-ff90-4deb-88f8-56a38484ad68 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:15.953203 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5384c5c-ff90-4deb-88f8-56a38484ad68 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:15.953203 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5384c5c-ff90-4deb-88f8-56a38484ad68 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:15.954583 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 641/1296] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:34:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:34:15.959887 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd382b8c-561a-442b-ad30-394ab2c87632 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:15.960316 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd382b8c-561a-442b-ad30-394ab2c87632 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:15.960577 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd382b8c-561a-442b-ad30-394ab2c87632 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:15.977447 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-fd382b8c-561a-442b-ad30-394ab2c87632 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:34:15.978655 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-fd382b8c-561a-442b-ad30-394ab2c87632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:16.052194 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 656/1297] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:34:15 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:34:16.075391 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5384c5c-ff90-4deb-88f8-56a38484ad68 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:16.175784 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5384c5c-ff90-4deb-88f8-56a38484ad68 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:16.208672 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d379735-630f-404f-b3f3-b3265c82baa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:16.209457 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d379735-630f-404f-b3f3-b3265c82baa0 None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:16.209743 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d379735-630f-404f-b3f3-b3265c82baa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:16.210018 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d379735-630f-404f-b3f3-b3265c82baa0 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:16.210664 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d379735-630f-404f-b3f3-b3265c82baa0 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:16.212048 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d379735-630f-404f-b3f3-b3265c82baa0 None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:16.224455 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-6d379735-630f-404f-b3f3-b3265c82baa0 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 17:34:16.224455 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:34:16.224455 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 17:34:16.224455 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 17:34:16.224455 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 17:34:16.224455 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 17:34:16.224455 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 17:34:16.224455 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:34:16.224455 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 17:34:16.224455 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:34:16.224455 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:34:16.224455 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:34:16.224455 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:34:16.224455 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:34:16.224455 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:34:16.224455 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:34:16.224455 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:34:16.225687 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:34:16.225687 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:34:16.225687 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:34:16.225687 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:34:16.225687 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:34:16.225687 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:34:16.225687 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 17:34:16.225687 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 17:34:16.225687 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 17:34:16.225687 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 17:34:16.225687 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:34:16.225687 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:34:16.225687 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:34:16.225687 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:34:16.225687 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:34:16.225687 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:34:16.226819 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:34:16.226819 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:34:16.226819 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:34:16.226819 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:34:16.226819 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:34:16.226819 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:34:16.226819 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:34:16.226819 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:34:16.226819 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:34:16.226819 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:34:16.226819 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 17:34:16.226819 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:34:16.226819 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 17:34:16.226819 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:34:16.226819 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 642/1298] 10.222.0.42 () {56 vars in 1117 bytes} [Thu Jun 16 17:34:16 2022] GET /identity/v3/services/placement => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:34:16.231707 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd382b8c-561a-442b-ad30-394ab2c87632 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:16.250262 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:34:16.330930 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd382b8c-561a-442b-ad30-394ab2c87632 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:16.365570 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5473b7-7e57-422a-99d1-ae3cd2c79fcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:16.366227 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5473b7-7e57-422a-99d1-ae3cd2c79fcf None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:16.366505 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5473b7-7e57-422a-99d1-ae3cd2c79fcf None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:16.366776 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5473b7-7e57-422a-99d1-ae3cd2c79fcf None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:16.367468 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5473b7-7e57-422a-99d1-ae3cd2c79fcf None admin] RBAC: Adding query filter params (name=placement) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:34:16.367772 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5473b7-7e57-422a-99d1-ae3cd2c79fcf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:16.369154 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5473b7-7e57-422a-99d1-ae3cd2c79fcf None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:16.374873 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 657/1299] 10.222.0.42 () {56 vars in 1116 bytes} [Thu Jun 16 17:34:16 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:34:16.380989 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d379735-630f-404f-b3f3-b3265c82baa0 None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:16.413064 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:34:16.480877 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d379735-630f-404f-b3f3-b3265c82baa0 None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:16.514050 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051d4ca0-8758-4fbb-895b-6307f263d73b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:16.514700 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051d4ca0-8758-4fbb-895b-6307f263d73b None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:16.514984 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051d4ca0-8758-4fbb-895b-6307f263d73b None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:16.515255 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051d4ca0-8758-4fbb-895b-6307f263d73b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:16.516030 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051d4ca0-8758-4fbb-895b-6307f263d73b None admin] RBAC: Adding query filter params (interface=public, service_id=1f1a31400b0043d0a4b2ef9b211685bd, region_id=RegionOne) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:34:16.516338 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051d4ca0-8758-4fbb-895b-6307f263d73b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:16.517718 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051d4ca0-8758-4fbb-895b-6307f263d73b None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:16.522858 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 643/1300] 10.222.0.42 () {56 vars in 1251 bytes} [Thu Jun 16 17:34:16 2022] GET /identity/v3/endpoints?service_id=1f1a31400b0043d0a4b2ef9b211685bd&interface=public®ion_id=RegionOne => generated 194 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:34:16.528577 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5473b7-7e57-422a-99d1-ae3cd2c79fcf None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:16.606854 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:34:16.625083 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5473b7-7e57-422a-99d1-ae3cd2c79fcf None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:16.658189 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195b903b-a708-457e-9f46-0f4602b2c064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:16.658830 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195b903b-a708-457e-9f46-0f4602b2c064 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:16.659110 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195b903b-a708-457e-9f46-0f4602b2c064 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:16.659383 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195b903b-a708-457e-9f46-0f4602b2c064 None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:16.660050 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195b903b-a708-457e-9f46-0f4602b2c064 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:16.661431 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195b903b-a708-457e-9f46-0f4602b2c064 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:16.668018 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 658/1301] 10.222.0.42 () {56 vars in 1087 bytes} [Thu Jun 16 17:34:16 2022] GET /identity/v3/services => generated 2191 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:34:18.868077 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18186cea-c004-4ad3-b52e-345463c2c085 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:18.868680 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18186cea-c004-4ad3-b52e-345463c2c085 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:18.868680 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18186cea-c004-4ad3-b52e-345463c2c085 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:18.870018 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 644/1302] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:34:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:34:18.875601 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a9cd29-d7d0-42c6-9a6c-1e327851b6ed None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:18.876040 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a9cd29-d7d0-42c6-9a6c-1e327851b6ed None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:18.876152 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a9cd29-d7d0-42c6-9a6c-1e327851b6ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:18.893123 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-a7a9cd29-d7d0-42c6-9a6c-1e327851b6ed None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:34:18.894296 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-a7a9cd29-d7d0-42c6-9a6c-1e327851b6ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:18.959045 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 659/1303] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:34:18 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:34:18.983499 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18186cea-c004-4ad3-b52e-345463c2c085 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:19.080434 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:34:19.087705 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18186cea-c004-4ad3-b52e-345463c2c085 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:19.120737 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51dd0e0-6106-4c9d-8ba3-6e5d953338da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:19.121464 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51dd0e0-6106-4c9d-8ba3-6e5d953338da None admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:19.121750 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51dd0e0-6106-4c9d-8ba3-6e5d953338da None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:19.122029 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51dd0e0-6106-4c9d-8ba3-6e5d953338da None admin] PATH_INFO: `/v3/services/placement` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:19.122681 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51dd0e0-6106-4c9d-8ba3-6e5d953338da None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:19.124050 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51dd0e0-6106-4c9d-8ba3-6e5d953338da None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:19.135714 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application [None req-b51dd0e0-6106-4c9d-8ba3-6e5d953338da None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 17:34:19.135714 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:34:19.135714 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 17:34:19.135714 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 17:34:19.135714 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 17:34:19.135714 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 17:34:19.135714 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 17:34:19.135714 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:34:19.135714 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 17:34:19.135714 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:34:19.135714 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:34:19.135714 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:34:19.135714 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:34:19.135714 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:34:19.135714 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:34:19.135714 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:34:19.135714 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:34:19.137028 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:34:19.137028 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:34:19.137028 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:34:19.137028 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:34:19.137028 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:34:19.137028 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:34:19.137028 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 17:34:19.137028 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 17:34:19.137028 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 17:34:19.137028 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 17:34:19.137028 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:34:19.137028 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:34:19.137028 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:34:19.137028 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:34:19.137028 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:34:19.137028 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:34:19.137968 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:34:19.137968 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application with Lock( Jun 16 17:34:19.137968 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:34:19.137968 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:34:19.137968 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:34:19.137968 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:34:19.137968 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:34:19.137968 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:34:19.137968 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:34:19.137968 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:34:19.137968 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 17:34:19.137968 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:34:19.137968 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 17:34:19.137968 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: ERROR keystone.server.flask.application Jun 16 17:34:19.137968 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 645/1304] 10.222.0.42 () {56 vars in 1117 bytes} [Thu Jun 16 17:34:18 2022] GET /identity/v3/services/placement => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:34:19.143607 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a9cd29-d7d0-42c6-9a6c-1e327851b6ed None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:19.219802 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:34:19.242565 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a9cd29-d7d0-42c6-9a6c-1e327851b6ed None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:19.276867 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba045e2a-3b03-442c-856f-3516650402fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:19.277525 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba045e2a-3b03-442c-856f-3516650402fb None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:19.277807 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba045e2a-3b03-442c-856f-3516650402fb None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:19.278082 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba045e2a-3b03-442c-856f-3516650402fb None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:19.278777 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba045e2a-3b03-442c-856f-3516650402fb None admin] RBAC: Adding query filter params (name=placement) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:34:19.279159 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba045e2a-3b03-442c-856f-3516650402fb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:19.280553 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba045e2a-3b03-442c-856f-3516650402fb None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:19.285550 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 660/1305] 10.222.0.42 () {56 vars in 1116 bytes} [Thu Jun 16 17:34:19 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:34:19.291329 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51dd0e0-6106-4c9d-8ba3-6e5d953338da None admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:19.387416 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51dd0e0-6106-4c9d-8ba3-6e5d953338da None admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:19.419952 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e71a741-4585-4e9e-b15e-b0276f7f7efc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:19.420613 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e71a741-4585-4e9e-b15e-b0276f7f7efc None admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:19.420890 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e71a741-4585-4e9e-b15e-b0276f7f7efc None admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:19.421161 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e71a741-4585-4e9e-b15e-b0276f7f7efc None admin] PATH_INFO: `/v3/endpoints` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:19.421891 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e71a741-4585-4e9e-b15e-b0276f7f7efc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:19.423243 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e71a741-4585-4e9e-b15e-b0276f7f7efc None admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:19.441752 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 646/1306] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Jun 16 17:34:19 2022] POST /identity/v3/endpoints => generated 331 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:34:29.282961 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=30 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:34:29.437711 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:35:23.191787 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61637c20-0efd-46f9-8ddd-ef43431927f9 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:23.191787 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61637c20-0efd-46f9-8ddd-ef43431927f9 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:23.192465 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61637c20-0efd-46f9-8ddd-ef43431927f9 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:23.197413 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 661/1307] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:35:23 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:35:23.198812 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958570aa-7016-4fa4-88ed-cadc441c37dc None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:23.199269 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958570aa-7016-4fa4-88ed-cadc441c37dc None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:23.199438 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958570aa-7016-4fa4-88ed-cadc441c37dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:23.281661 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-958570aa-7016-4fa4-88ed-cadc441c37dc None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:35:23.282904 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-958570aa-7016-4fa4-88ed-cadc441c37dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:23.407877 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 647/1308] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:35:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:35:23.463731 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1776ff-fd96-4bb2-aace-5d4b53dd4aeb None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:23.464075 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1776ff-fd96-4bb2-aace-5d4b53dd4aeb None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:23.464449 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1776ff-fd96-4bb2-aace-5d4b53dd4aeb None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:23.466998 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 662/1309] 10.222.0.42 () {54 vars in 964 bytes} [Thu Jun 16 17:35:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:35:23.474741 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83af2ea-f40a-485f-a07c-a9e4b6edef1e None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:23.475106 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83af2ea-f40a-485f-a07c-a9e4b6edef1e None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:23.475453 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83af2ea-f40a-485f-a07c-a9e4b6edef1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:23.567446 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-c83af2ea-f40a-485f-a07c-a9e4b6edef1e None None] MFA Rules not processed for user `f94416103f6245888214a997b014f47a`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:35:23.569354 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-c83af2ea-f40a-485f-a07c-a9e4b6edef1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:23.595178 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:35:23.672176 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1776ff-fd96-4bb2-aace-5d4b53dd4aeb None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:23.674188 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-3d1776ff-fd96-4bb2-aace-5d4b53dd4aeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:23.683342 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 648/1310] 10.222.0.42 () {58 vars in 1061 bytes} [Thu Jun 16 17:35:23 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:35:23.811345 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1776ff-fd96-4bb2-aace-5d4b53dd4aeb None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:23.896757 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6111f3cd-19e0-4b96-8f56-2d5f19265820 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:23.897666 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6111f3cd-19e0-4b96-8f56-2d5f19265820 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:23.897950 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6111f3cd-19e0-4b96-8f56-2d5f19265820 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:23.898226 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6111f3cd-19e0-4b96-8f56-2d5f19265820 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:23.899877 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-6111f3cd-19e0-4b96-8f56-2d5f19265820 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:23.961042 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:35:24.018854 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6111f3cd-19e0-4b96-8f56-2d5f19265820 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:24.020300 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6111f3cd-19e0-4b96-8f56-2d5f19265820 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:24.040420 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 663/1311] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:35:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 17:35:26.541652 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae64227-eaaf-41fb-88c8-996814c82d49 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:26.542183 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae64227-eaaf-41fb-88c8-996814c82d49 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:26.542274 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae64227-eaaf-41fb-88c8-996814c82d49 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:26.543601 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 649/1312] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:35:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:35:26.548850 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5f4483-67a0-4806-b9a9-6d5120ddb48b None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:26.549273 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5f4483-67a0-4806-b9a9-6d5120ddb48b None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:26.549552 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5f4483-67a0-4806-b9a9-6d5120ddb48b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:26.629820 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-7c5f4483-67a0-4806-b9a9-6d5120ddb48b None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:35:26.631047 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-7c5f4483-67a0-4806-b9a9-6d5120ddb48b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:26.731785 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 664/1313] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:35:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:35:26.771708 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae64227-eaaf-41fb-88c8-996814c82d49 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:26.773577 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-aae64227-eaaf-41fb-88c8-996814c82d49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:26.891915 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae64227-eaaf-41fb-88c8-996814c82d49 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:26.968199 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:35:26.970954 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793a614b-5dba-450f-acf4-504a488b257f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:26.971987 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793a614b-5dba-450f-acf4-504a488b257f service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:26.972489 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793a614b-5dba-450f-acf4-504a488b257f service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:26.972972 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793a614b-5dba-450f-acf4-504a488b257f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:26.974724 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-793a614b-5dba-450f-acf4-504a488b257f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:27.086449 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793a614b-5dba-450f-acf4-504a488b257f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:27.087920 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793a614b-5dba-450f-acf4-504a488b257f service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:27.105945 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 650/1314] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:35:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:35:29.629683 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923e05f0-ebd3-4495-84ec-4166cdca6a10 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:29.630196 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923e05f0-ebd3-4495-84ec-4166cdca6a10 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:29.630196 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923e05f0-ebd3-4495-84ec-4166cdca6a10 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:29.631529 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 665/1315] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:35:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:35:29.636999 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383e2899-182e-48fe-bc1c-e6cd9e896c36 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:29.637408 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383e2899-182e-48fe-bc1c-e6cd9e896c36 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:29.638222 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383e2899-182e-48fe-bc1c-e6cd9e896c36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:29.720132 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-383e2899-182e-48fe-bc1c-e6cd9e896c36 None None] MFA Rules not processed for user `f72678162d55418190ac5579cb1eb41f`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:35:29.721427 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-383e2899-182e-48fe-bc1c-e6cd9e896c36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:29.785592 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:35:29.835938 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 651/1316] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:35:29 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:35:29.896506 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dad6c63-d57e-4602-9673-c426c1089dc3 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:29.896808 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dad6c63-d57e-4602-9673-c426c1089dc3 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:29.897093 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dad6c63-d57e-4602-9673-c426c1089dc3 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:29.898732 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 666/1317] 10.222.0.42 () {54 vars in 964 bytes} [Thu Jun 16 17:35:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:35:29.906124 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266072e6-8b7b-482b-9b5d-4c4f708b63c9 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:29.906415 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266072e6-8b7b-482b-9b5d-4c4f708b63c9 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:29.906701 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266072e6-8b7b-482b-9b5d-4c4f708b63c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:29.996237 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-266072e6-8b7b-482b-9b5d-4c4f708b63c9 None None] MFA Rules not processed for user `f94416103f6245888214a997b014f47a`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:35:29.997692 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-266072e6-8b7b-482b-9b5d-4c4f708b63c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:30.071625 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 652/1318] 10.222.0.42 () {58 vars in 1061 bytes} [Thu Jun 16 17:35:29 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:35:30.080884 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dad6c63-d57e-4602-9673-c426c1089dc3 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:30.082460 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-1dad6c63-d57e-4602-9673-c426c1089dc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:30.092744 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:35:30.186695 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dad6c63-d57e-4602-9673-c426c1089dc3 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:30.259117 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc14b2a-e097-4a9d-99f8-05288725a21a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:30.259764 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc14b2a-e097-4a9d-99f8-05288725a21a service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:30.260049 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc14b2a-e097-4a9d-99f8-05288725a21a service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:30.260327 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc14b2a-e097-4a9d-99f8-05288725a21a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:30.261996 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-0dc14b2a-e097-4a9d-99f8-05288725a21a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:30.337010 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:35:30.384895 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc14b2a-e097-4a9d-99f8-05288725a21a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:30.386335 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc14b2a-e097-4a9d-99f8-05288725a21a service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:30.406371 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 667/1319] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:35:30 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:35:33.479995 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ee8583-2dbf-4f8e-b857-ce8907b43bd3 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:33.480554 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ee8583-2dbf-4f8e-b857-ce8907b43bd3 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:33.480554 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ee8583-2dbf-4f8e-b857-ce8907b43bd3 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:33.482515 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 653/1320] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:35:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:35:33.487638 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a549eb86-9553-447c-a190-7a03cb5e7d36 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:33.488073 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a549eb86-9553-447c-a190-7a03cb5e7d36 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:33.488199 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a549eb86-9553-447c-a190-7a03cb5e7d36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:33.552806 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-a549eb86-9553-447c-a190-7a03cb5e7d36 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:35:33.553962 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-a549eb86-9553-447c-a190-7a03cb5e7d36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:33.650388 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 668/1321] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:35:33 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:35:33.695875 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ee8583-2dbf-4f8e-b857-ce8907b43bd3 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:33.697425 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-29ee8583-2dbf-4f8e-b857-ce8907b43bd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:33.750232 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:35:33.804264 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ee8583-2dbf-4f8e-b857-ce8907b43bd3 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:33.876639 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d9e603-701e-49bd-a6aa-75ff21de753c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:33.877287 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d9e603-701e-49bd-a6aa-75ff21de753c service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:33.877567 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d9e603-701e-49bd-a6aa-75ff21de753c service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:33.877841 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d9e603-701e-49bd-a6aa-75ff21de753c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:33.879466 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-a6d9e603-701e-49bd-a6aa-75ff21de753c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:33.976078 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:35:33.989724 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d9e603-701e-49bd-a6aa-75ff21de753c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:33.991161 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d9e603-701e-49bd-a6aa-75ff21de753c service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:34.008865 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 654/1322] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:35:33 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:35:36.830079 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e5ec83-5a62-44fe-978c-c17db552fd0e None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:36.830621 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e5ec83-5a62-44fe-978c-c17db552fd0e None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:36.830621 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e5ec83-5a62-44fe-978c-c17db552fd0e None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:36.831995 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 669/1323] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:35:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:35:36.837127 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d11abc-0b27-4039-9f63-5437670ca7ea None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:36.837572 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d11abc-0b27-4039-9f63-5437670ca7ea None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:36.837760 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d11abc-0b27-4039-9f63-5437670ca7ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:36.905691 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-70d11abc-0b27-4039-9f63-5437670ca7ea None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:35:36.906946 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-70d11abc-0b27-4039-9f63-5437670ca7ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:37.012643 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 655/1324] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:35:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:35:37.050333 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e5ec83-5a62-44fe-978c-c17db552fd0e None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:37.051983 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-26e5ec83-5a62-44fe-978c-c17db552fd0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:37.074287 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:35:37.177656 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e5ec83-5a62-44fe-978c-c17db552fd0e None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:37.253082 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d154a6a-cf00-4a1e-adca-125efc008a72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:37.253738 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d154a6a-cf00-4a1e-adca-125efc008a72 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:37.254024 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d154a6a-cf00-4a1e-adca-125efc008a72 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:37.254303 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d154a6a-cf00-4a1e-adca-125efc008a72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:37.255954 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-7d154a6a-cf00-4a1e-adca-125efc008a72 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:37.328823 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:35:37.373791 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d154a6a-cf00-4a1e-adca-125efc008a72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:37.375052 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d154a6a-cf00-4a1e-adca-125efc008a72 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:37.394129 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 670/1325] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:35:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:35:39.629216 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ef5a9d-0302-466d-87e5-4fc89cce00e7 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:39.629748 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ef5a9d-0302-466d-87e5-4fc89cce00e7 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:39.629748 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ef5a9d-0302-466d-87e5-4fc89cce00e7 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:39.631064 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 656/1326] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:35:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:35:39.636338 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389fcb3a-48b6-4b4c-902c-351849c8aa98 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:39.636786 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389fcb3a-48b6-4b4c-902c-351849c8aa98 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:39.636920 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389fcb3a-48b6-4b4c-902c-351849c8aa98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:39.702373 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-389fcb3a-48b6-4b4c-902c-351849c8aa98 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:35:39.703596 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-389fcb3a-48b6-4b4c-902c-351849c8aa98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:39.802621 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 671/1327] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:35:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:35:39.838369 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ef5a9d-0302-466d-87e5-4fc89cce00e7 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:39.839888 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-b7ef5a9d-0302-466d-87e5-4fc89cce00e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:39.906005 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:35:39.951222 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ef5a9d-0302-466d-87e5-4fc89cce00e7 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:40.031400 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f07e871-9bde-454c-abe6-0e151119b14b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:40.032047 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f07e871-9bde-454c-abe6-0e151119b14b service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:40.032326 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f07e871-9bde-454c-abe6-0e151119b14b service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:40.032629 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f07e871-9bde-454c-abe6-0e151119b14b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:40.034247 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-8f07e871-9bde-454c-abe6-0e151119b14b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:40.141198 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:35:40.143872 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f07e871-9bde-454c-abe6-0e151119b14b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:40.145643 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f07e871-9bde-454c-abe6-0e151119b14b service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:40.168684 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 657/1328] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:35:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:35:42.374726 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c870f7-6e33-423e-bde9-8cd82e916004 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:42.375269 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c870f7-6e33-423e-bde9-8cd82e916004 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:42.375269 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c870f7-6e33-423e-bde9-8cd82e916004 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:42.376692 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 672/1329] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:35:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:35:42.382164 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa8c1af-00c8-4423-8c10-f51eef559830 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:42.382590 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa8c1af-00c8-4423-8c10-f51eef559830 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:42.382719 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa8c1af-00c8-4423-8c10-f51eef559830 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:42.449339 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-cfa8c1af-00c8-4423-8c10-f51eef559830 None None] MFA Rules not processed for user `f72678162d55418190ac5579cb1eb41f`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:35:42.451270 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-cfa8c1af-00c8-4423-8c10-f51eef559830 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:42.549926 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 658/1330] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:35:42 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:35:42.587312 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c870f7-6e33-423e-bde9-8cd82e916004 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:42.589111 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-50c870f7-6e33-423e-bde9-8cd82e916004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:42.620310 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:35:42.695226 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c870f7-6e33-423e-bde9-8cd82e916004 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:42.782372 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7708de4-fff1-45d6-9af5-f06835bb78ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:42.783023 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7708de4-fff1-45d6-9af5-f06835bb78ec service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:42.783315 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7708de4-fff1-45d6-9af5-f06835bb78ec service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:42.783579 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7708de4-fff1-45d6-9af5-f06835bb78ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:42.787646 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-d7708de4-fff1-45d6-9af5-f06835bb78ec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:42.883493 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:35:42.916501 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7708de4-fff1-45d6-9af5-f06835bb78ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:42.917946 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7708de4-fff1-45d6-9af5-f06835bb78ec service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:42.937102 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 673/1331] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:35:42 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:35:45.379680 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dedf92-9b4a-4c9f-8f10-c3e629406905 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:45.380226 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dedf92-9b4a-4c9f-8f10-c3e629406905 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:45.380226 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dedf92-9b4a-4c9f-8f10-c3e629406905 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:45.381721 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 659/1332] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:35:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:35:45.387355 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1925289-0bac-4ea9-b8b9-4dfcafbc5b2a None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:45.387844 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1925289-0bac-4ea9-b8b9-4dfcafbc5b2a None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:45.388630 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1925289-0bac-4ea9-b8b9-4dfcafbc5b2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:45.454581 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-b1925289-0bac-4ea9-b8b9-4dfcafbc5b2a None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:35:45.456049 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-b1925289-0bac-4ea9-b8b9-4dfcafbc5b2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:45.555534 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 674/1333] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:35:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:35:45.591433 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dedf92-9b4a-4c9f-8f10-c3e629406905 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:45.592954 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-e4dedf92-9b4a-4c9f-8f10-c3e629406905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:45.661924 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:35:45.696644 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dedf92-9b4a-4c9f-8f10-c3e629406905 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:45.769452 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c10ea4-8c43-4079-8ce8-60ccf24e31b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:45.770104 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c10ea4-8c43-4079-8ce8-60ccf24e31b0 service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:45.770387 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c10ea4-8c43-4079-8ce8-60ccf24e31b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:45.770663 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c10ea4-8c43-4079-8ce8-60ccf24e31b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:45.772276 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-a6c10ea4-8c43-4079-8ce8-60ccf24e31b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:45.908546 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c10ea4-8c43-4079-8ce8-60ccf24e31b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:45.909990 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c10ea4-8c43-4079-8ce8-60ccf24e31b0 service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:45.920444 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:35:45.929002 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 660/1334] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:35:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:35:48.129533 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee4a565-1716-418a-a07a-f8a5c4d976e7 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:48.130064 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee4a565-1716-418a-a07a-f8a5c4d976e7 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:48.130064 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee4a565-1716-418a-a07a-f8a5c4d976e7 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:48.131433 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 675/1335] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:35:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:35:48.136562 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fa05c7-b3ba-4e0b-b57c-1f6ba575df5a None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:48.136965 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fa05c7-b3ba-4e0b-b57c-1f6ba575df5a None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:48.137059 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fa05c7-b3ba-4e0b-b57c-1f6ba575df5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:48.203972 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-30fa05c7-b3ba-4e0b-b57c-1f6ba575df5a None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:35:48.205214 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-30fa05c7-b3ba-4e0b-b57c-1f6ba575df5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:48.313940 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 661/1336] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:35:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:35:48.353715 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee4a565-1716-418a-a07a-f8a5c4d976e7 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:48.355248 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-8ee4a565-1716-418a-a07a-f8a5c4d976e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:48.397258 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:35:48.462212 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee4a565-1716-418a-a07a-f8a5c4d976e7 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:48.536840 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b50bc2-8f44-48f9-86d3-7241950fb006 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:48.537479 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b50bc2-8f44-48f9-86d3-7241950fb006 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:48.537756 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b50bc2-8f44-48f9-86d3-7241950fb006 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:48.538025 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b50bc2-8f44-48f9-86d3-7241950fb006 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:48.539644 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-d2b50bc2-8f44-48f9-86d3-7241950fb006 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:48.625126 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:35:48.651937 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b50bc2-8f44-48f9-86d3-7241950fb006 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:48.654199 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b50bc2-8f44-48f9-86d3-7241950fb006 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:48.671959 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 676/1337] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:35:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:35:51.203946 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252f6c5b-f443-4fa5-a78b-97618e63ac2f None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:51.204570 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252f6c5b-f443-4fa5-a78b-97618e63ac2f None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:51.204570 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252f6c5b-f443-4fa5-a78b-97618e63ac2f None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:51.205923 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 662/1338] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:35:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:35:51.211177 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0513f48f-8144-4f4b-851b-a93642778305 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:51.211576 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0513f48f-8144-4f4b-851b-a93642778305 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:51.211747 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0513f48f-8144-4f4b-851b-a93642778305 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:51.277117 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-0513f48f-8144-4f4b-851b-a93642778305 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:35:51.278532 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-0513f48f-8144-4f4b-851b-a93642778305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:51.381856 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 677/1339] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:35:51 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:35:51.423471 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252f6c5b-f443-4fa5-a78b-97618e63ac2f None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:51.425048 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-252f6c5b-f443-4fa5-a78b-97618e63ac2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:51.501266 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:35:51.530801 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252f6c5b-f443-4fa5-a78b-97618e63ac2f None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:51.603695 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed273885-45de-4d98-9b10-7d41e9ee83f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:51.604337 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed273885-45de-4d98-9b10-7d41e9ee83f7 service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:51.604642 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed273885-45de-4d98-9b10-7d41e9ee83f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:51.604913 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed273885-45de-4d98-9b10-7d41e9ee83f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:51.607084 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-ed273885-45de-4d98-9b10-7d41e9ee83f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:51.723153 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed273885-45de-4d98-9b10-7d41e9ee83f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:51.724612 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed273885-45de-4d98-9b10-7d41e9ee83f7 service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:51.742741 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 663/1340] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:35:51 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:35:56.106366 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5b608e-f7d8-4d33-a657-7c01225f42e7 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:56.106895 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5b608e-f7d8-4d33-a657-7c01225f42e7 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:56.106895 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5b608e-f7d8-4d33-a657-7c01225f42e7 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:56.108254 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 678/1341] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:35:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:35:56.113526 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3152d4a2-7eef-471c-b1f7-ea4ae960cb12 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:56.113961 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3152d4a2-7eef-471c-b1f7-ea4ae960cb12 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:56.114213 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3152d4a2-7eef-471c-b1f7-ea4ae960cb12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:56.120273 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:35:56.186401 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-3152d4a2-7eef-471c-b1f7-ea4ae960cb12 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:35:56.187592 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-3152d4a2-7eef-471c-b1f7-ea4ae960cb12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:56.289520 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 664/1342] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:35:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:35:56.328705 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5b608e-f7d8-4d33-a657-7c01225f42e7 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:56.330875 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-5e5b608e-f7d8-4d33-a657-7c01225f42e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:56.384296 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:35:56.433369 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5b608e-f7d8-4d33-a657-7c01225f42e7 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:56.437565 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3a47dd-10e0-43cb-aad6-2f6a61e516a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:56.438210 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3a47dd-10e0-43cb-aad6-2f6a61e516a7 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:56.438490 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3a47dd-10e0-43cb-aad6-2f6a61e516a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:56.438762 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3a47dd-10e0-43cb-aad6-2f6a61e516a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:56.447880 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3a47dd-10e0-43cb-aad6-2f6a61e516a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:56.449383 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3a47dd-10e0-43cb-aad6-2f6a61e516a7 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:56.459650 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 679/1343] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:35:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:35:59.225538 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6d88f3-ff8c-4fc9-819f-f39550212094 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:59.226106 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6d88f3-ff8c-4fc9-819f-f39550212094 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:59.226106 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6d88f3-ff8c-4fc9-819f-f39550212094 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:59.227477 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 665/1344] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:35:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:35:59.232684 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f9c8f1-1185-4e2f-829c-81d1d6b5b4f3 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:59.233119 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f9c8f1-1185-4e2f-829c-81d1d6b5b4f3 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:59.233316 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f9c8f1-1185-4e2f-829c-81d1d6b5b4f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:59.265346 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-82f9c8f1-1185-4e2f-829c-81d1d6b5b4f3 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:35:59.266593 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-82f9c8f1-1185-4e2f-829c-81d1d6b5b4f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:59.350982 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 680/1345] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:35:59 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:35:59.391547 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6d88f3-ff8c-4fc9-819f-f39550212094 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:59.406065 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6d88f3-ff8c-4fc9-819f-f39550212094 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:59.409578 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9505a186-05ce-4471-90a0-2326911dc543 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:59.410301 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9505a186-05ce-4471-90a0-2326911dc543 service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:59.410544 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9505a186-05ce-4471-90a0-2326911dc543 service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:59.410802 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9505a186-05ce-4471-90a0-2326911dc543 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:59.420422 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9505a186-05ce-4471-90a0-2326911dc543 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:59.421868 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9505a186-05ce-4471-90a0-2326911dc543 service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:59.432197 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 666/1346] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:35:59 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:36:03.849219 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f8036e-ab1c-4b52-8adc-48de56a42297 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:03.849778 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f8036e-ab1c-4b52-8adc-48de56a42297 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:03.849778 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f8036e-ab1c-4b52-8adc-48de56a42297 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:03.851203 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 681/1347] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:36:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:36:03.857685 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fc3d3b-3df6-4f53-ab68-d324b93b7508 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:03.858146 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fc3d3b-3df6-4f53-ab68-d324b93b7508 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:03.858276 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fc3d3b-3df6-4f53-ab68-d324b93b7508 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:03.884596 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-e5fc3d3b-3df6-4f53-ab68-d324b93b7508 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:36:03.886021 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-e5fc3d3b-3df6-4f53-ab68-d324b93b7508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:36:03.897170 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 667/1348] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:36:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:36:03.938178 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f8036e-ab1c-4b52-8adc-48de56a42297 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:03.950362 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f8036e-ab1c-4b52-8adc-48de56a42297 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:03.954920 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c862cd7a-7a34-4b53-8920-ca983f051b39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:03.955568 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c862cd7a-7a34-4b53-8920-ca983f051b39 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:03.955849 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c862cd7a-7a34-4b53-8920-ca983f051b39 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:03.956125 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c862cd7a-7a34-4b53-8920-ca983f051b39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:03.982797 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c862cd7a-7a34-4b53-8920-ca983f051b39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:03.986712 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c862cd7a-7a34-4b53-8920-ca983f051b39 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:03.998724 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 682/1349] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:36:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:36:09.118032 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782a0264-0fb8-4927-8662-58c29c49eae5 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:09.118990 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782a0264-0fb8-4927-8662-58c29c49eae5 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:09.119483 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782a0264-0fb8-4927-8662-58c29c49eae5 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:09.121577 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 668/1350] 10.222.0.42 () {54 vars in 921 bytes} [Thu Jun 16 17:36:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:36:09.130766 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b5dde4-1cdc-4e55-8395-25368da36c31 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:09.131228 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b5dde4-1cdc-4e55-8395-25368da36c31 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:09.131228 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b5dde4-1cdc-4e55-8395-25368da36c31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:09.248060 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-18b5dde4-1cdc-4e55-8395-25368da36c31 None None] MFA Rules not processed for user `0570d8d9508d4546ae897276f7d181d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:36:09.249646 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-18b5dde4-1cdc-4e55-8395-25368da36c31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:36:09.288703 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:36:09.330321 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 683/1351] 10.222.0.42 () {58 vars in 1018 bytes} [Thu Jun 16 17:36:09 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:36:13.647522 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e4d380-8b0f-4913-9244-e2e281e20ad5 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:13.648029 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e4d380-8b0f-4913-9244-e2e281e20ad5 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:13.648029 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e4d380-8b0f-4913-9244-e2e281e20ad5 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:13.649740 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 669/1352] 10.222.0.42 () {54 vars in 921 bytes} [Thu Jun 16 17:36:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:36:13.655761 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaa6dad-65bf-4d41-8b9f-ff6ba1440c98 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:13.656687 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaa6dad-65bf-4d41-8b9f-ff6ba1440c98 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:13.657052 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaa6dad-65bf-4d41-8b9f-ff6ba1440c98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:13.677340 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-dfaa6dad-65bf-4d41-8b9f-ff6ba1440c98 None None] MFA Rules not processed for user `0570d8d9508d4546ae897276f7d181d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:36:13.678579 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-dfaa6dad-65bf-4d41-8b9f-ff6ba1440c98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:36:13.688539 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 684/1353] 10.222.0.42 () {58 vars in 1018 bytes} [Thu Jun 16 17:36:13 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:36:13.871026 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=72 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:36:15.223360 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8df04c-97af-4cb6-98d1-a744ab958dab None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:15.223981 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8df04c-97af-4cb6-98d1-a744ab958dab None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:15.223981 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8df04c-97af-4cb6-98d1-a744ab958dab None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:15.225286 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 670/1354] 10.222.0.42 () {54 vars in 921 bytes} [Thu Jun 16 17:36:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:36:15.231172 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a54925c-2042-46e4-a1be-6717aeb15a62 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:15.231641 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a54925c-2042-46e4-a1be-6717aeb15a62 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:15.231877 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a54925c-2042-46e4-a1be-6717aeb15a62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:15.250232 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-1a54925c-2042-46e4-a1be-6717aeb15a62 None None] MFA Rules not processed for user `0570d8d9508d4546ae897276f7d181d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:36:15.251426 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-1a54925c-2042-46e4-a1be-6717aeb15a62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:36:15.260610 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 685/1355] 10.222.0.42 () {58 vars in 1018 bytes} [Thu Jun 16 17:36:15 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:36:19.057717 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a468bc-83a3-432e-b262-a924fae651c4 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:19.058376 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a468bc-83a3-432e-b262-a924fae651c4 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:19.058662 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a468bc-83a3-432e-b262-a924fae651c4 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:19.061348 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 671/1356] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:36:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:36:19.065456 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e056468-da27-4622-87bf-539a34782b73 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:19.065868 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e056468-da27-4622-87bf-539a34782b73 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:19.066012 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e056468-da27-4622-87bf-539a34782b73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:19.086983 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-7e056468-da27-4622-87bf-539a34782b73 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:36:19.089317 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-7e056468-da27-4622-87bf-539a34782b73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:36:19.097609 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 686/1357] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:36:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:36:19.153370 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e51fe6-bd7c-4cac-a113-4795fdc58a3c None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:19.153664 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e51fe6-bd7c-4cac-a113-4795fdc58a3c None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:19.153945 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e51fe6-bd7c-4cac-a113-4795fdc58a3c None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:19.155391 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 672/1358] 10.222.0.42 () {54 vars in 961 bytes} [Thu Jun 16 17:36:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:36:19.166692 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015752c8-b84e-49f1-beaf-426699bf5e2a None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:19.167206 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015752c8-b84e-49f1-beaf-426699bf5e2a None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:19.167579 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015752c8-b84e-49f1-beaf-426699bf5e2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:19.224203 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-015752c8-b84e-49f1-beaf-426699bf5e2a None None] MFA Rules not processed for user `f86c16179a70474a817be9fda33ece9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:36:19.225557 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-015752c8-b84e-49f1-beaf-426699bf5e2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:36:19.316620 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 687/1359] 10.222.0.42 () {58 vars in 1058 bytes} [Thu Jun 16 17:36:19 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:36:19.325010 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e51fe6-bd7c-4cac-a113-4795fdc58a3c None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:19.339731 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e51fe6-bd7c-4cac-a113-4795fdc58a3c None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:19.345221 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec17b58-a11f-4c1c-975a-3a463f2a9dcf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f86c16179a70474a817be9fda33ece9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:19.345859 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec17b58-a11f-4c1c-975a-3a463f2a9dcf service nova] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:19.346140 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec17b58-a11f-4c1c-975a-3a463f2a9dcf service nova] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:19.346414 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec17b58-a11f-4c1c-975a-3a463f2a9dcf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:19.356219 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec17b58-a11f-4c1c-975a-3a463f2a9dcf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:19.357923 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec17b58-a11f-4c1c-975a-3a463f2a9dcf service nova] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:19.392853 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 673/1360] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:36:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:36:21.746691 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1face39-40d1-4299-9ebb-d88f8318e800 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:21.747269 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1face39-40d1-4299-9ebb-d88f8318e800 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:21.747269 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1face39-40d1-4299-9ebb-d88f8318e800 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:21.748829 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 688/1361] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:36:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:36:21.754357 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0375d22-b23f-4aae-af3e-46aeca9876de None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:21.754890 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0375d22-b23f-4aae-af3e-46aeca9876de None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:21.754890 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0375d22-b23f-4aae-af3e-46aeca9876de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:21.775897 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-d0375d22-b23f-4aae-af3e-46aeca9876de None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:36:21.777464 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-d0375d22-b23f-4aae-af3e-46aeca9876de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:36:21.786649 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 674/1362] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:36:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:36:21.835252 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1face39-40d1-4299-9ebb-d88f8318e800 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:21.851350 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1face39-40d1-4299-9ebb-d88f8318e800 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:21.863887 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586a8858-e3a7-40ae-80fe-f0de0aba98ba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f86c16179a70474a817be9fda33ece9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:21.864685 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586a8858-e3a7-40ae-80fe-f0de0aba98ba service nova] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:21.865156 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586a8858-e3a7-40ae-80fe-f0de0aba98ba service nova] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:21.865501 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586a8858-e3a7-40ae-80fe-f0de0aba98ba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:21.877733 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586a8858-e3a7-40ae-80fe-f0de0aba98ba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:21.879431 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586a8858-e3a7-40ae-80fe-f0de0aba98ba service nova] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:21.891872 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 689/1363] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:36:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:36:24.421057 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d458e6-ceb6-4386-8623-af97cf7a481a None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:24.421557 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d458e6-ceb6-4386-8623-af97cf7a481a None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:24.421664 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d458e6-ceb6-4386-8623-af97cf7a481a None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:24.423008 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 675/1364] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:36:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:36:24.428145 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cefdabc-defe-494b-850b-75f5b55e7590 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:24.428569 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cefdabc-defe-494b-850b-75f5b55e7590 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:24.428734 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cefdabc-defe-494b-850b-75f5b55e7590 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:24.449717 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-4cefdabc-defe-494b-850b-75f5b55e7590 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:36:24.450957 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-4cefdabc-defe-494b-850b-75f5b55e7590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:36:24.458823 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 690/1365] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:36:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:36:24.504949 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e097c0e-d6d8-4fbe-ad5c-3b6cea636a0a None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:24.505241 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e097c0e-d6d8-4fbe-ad5c-3b6cea636a0a None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:24.505522 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e097c0e-d6d8-4fbe-ad5c-3b6cea636a0a None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:24.506903 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 676/1366] 10.222.0.42 () {54 vars in 961 bytes} [Thu Jun 16 17:36:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:36:24.512928 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4599eee1-230d-4a8b-861f-3bb2fec3f562 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:24.513245 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4599eee1-230d-4a8b-861f-3bb2fec3f562 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:24.513527 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4599eee1-230d-4a8b-861f-3bb2fec3f562 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:24.538781 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-4599eee1-230d-4a8b-861f-3bb2fec3f562 None None] MFA Rules not processed for user `f86c16179a70474a817be9fda33ece9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:36:24.539629 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-4599eee1-230d-4a8b-861f-3bb2fec3f562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:36:24.550689 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 691/1367] 10.222.0.42 () {58 vars in 1058 bytes} [Thu Jun 16 17:36:24 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:36:24.559304 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e097c0e-d6d8-4fbe-ad5c-3b6cea636a0a None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:24.574213 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e097c0e-d6d8-4fbe-ad5c-3b6cea636a0a None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:24.580358 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce16493-f640-4a09-bda7-73c3ea82279d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f86c16179a70474a817be9fda33ece9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:24.581253 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce16493-f640-4a09-bda7-73c3ea82279d service nova] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:24.581771 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce16493-f640-4a09-bda7-73c3ea82279d service nova] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:24.582270 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce16493-f640-4a09-bda7-73c3ea82279d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:24.592548 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce16493-f640-4a09-bda7-73c3ea82279d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:24.594180 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce16493-f640-4a09-bda7-73c3ea82279d service nova] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:24.605965 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 677/1368] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:36:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:36:26.853503 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b31758a-7975-4488-b6c6-39996d4a3a69 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:26.854095 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b31758a-7975-4488-b6c6-39996d4a3a69 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:26.854095 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b31758a-7975-4488-b6c6-39996d4a3a69 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:26.855439 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 692/1369] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:36:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:36:26.861137 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c931e7-6cb9-46b2-8ae0-49693c7a8815 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:26.861936 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c931e7-6cb9-46b2-8ae0-49693c7a8815 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:26.862455 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c931e7-6cb9-46b2-8ae0-49693c7a8815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:26.881548 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-17c931e7-6cb9-46b2-8ae0-49693c7a8815 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:36:26.882999 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-17c931e7-6cb9-46b2-8ae0-49693c7a8815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:36:26.892346 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 678/1370] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:36:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:36:26.939407 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b31758a-7975-4488-b6c6-39996d4a3a69 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:26.955519 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b31758a-7975-4488-b6c6-39996d4a3a69 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:26.961485 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a712990f-756b-4591-bada-ea77f59a9a42 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f86c16179a70474a817be9fda33ece9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:26.962400 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a712990f-756b-4591-bada-ea77f59a9a42 service nova] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:26.962871 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a712990f-756b-4591-bada-ea77f59a9a42 service nova] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:26.963326 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a712990f-756b-4591-bada-ea77f59a9a42 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:26.972888 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a712990f-756b-4591-bada-ea77f59a9a42 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:26.974693 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a712990f-756b-4591-bada-ea77f59a9a42 service nova] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:26.986392 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 693/1371] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:36:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:36:27.332027 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ad5e92-a39c-4e3a-a1d0-56eb51b47830 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:27.332300 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ad5e92-a39c-4e3a-a1d0-56eb51b47830 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:27.332808 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ad5e92-a39c-4e3a-a1d0-56eb51b47830 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:27.334195 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 679/1372] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:36:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:36:27.339786 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f94fea2-245f-47fb-8741-aa089ed0f2f3 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:27.340189 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f94fea2-245f-47fb-8741-aa089ed0f2f3 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:27.340612 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f94fea2-245f-47fb-8741-aa089ed0f2f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:27.360313 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-3f94fea2-245f-47fb-8741-aa089ed0f2f3 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:36:27.361700 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-3f94fea2-245f-47fb-8741-aa089ed0f2f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:36:27.372421 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 694/1373] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:36:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:36:27.674765 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ae3616-b980-40bf-8e36-35627751a1b2 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:27.674765 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ae3616-b980-40bf-8e36-35627751a1b2 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:27.674765 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ae3616-b980-40bf-8e36-35627751a1b2 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:27.677604 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 680/1374] 10.222.0.42 () {54 vars in 963 bytes} [Thu Jun 16 17:36:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:36:27.682001 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37a9aa8-8284-4378-84a8-5893b6fd0c0c None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:27.682258 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37a9aa8-8284-4378-84a8-5893b6fd0c0c None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:27.682449 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37a9aa8-8284-4378-84a8-5893b6fd0c0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:27.689856 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:36:27.735405 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-a37a9aa8-8284-4378-84a8-5893b6fd0c0c None None] MFA Rules not processed for user `24d5e2637b7043e08313909d5b06e329`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:36:27.736657 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-a37a9aa8-8284-4378-84a8-5893b6fd0c0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:36:27.801158 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 695/1375] 10.222.0.42 () {58 vars in 1060 bytes} [Thu Jun 16 17:36:27 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:36:27.809174 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ae3616-b980-40bf-8e36-35627751a1b2 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:27.820092 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ae3616-b980-40bf-8e36-35627751a1b2 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:27.824928 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2396dfe-f8e8-4866-8e4a-760b57dd8ffb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24d5e2637b7043e08313909d5b06e329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:27.825580 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2396dfe-f8e8-4866-8e4a-760b57dd8ffb service cinder] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:27.825867 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2396dfe-f8e8-4866-8e4a-760b57dd8ffb service cinder] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:27.826144 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2396dfe-f8e8-4866-8e4a-760b57dd8ffb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:27.835860 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2396dfe-f8e8-4866-8e4a-760b57dd8ffb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:27.837547 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2396dfe-f8e8-4866-8e4a-760b57dd8ffb service cinder] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:27.849270 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 681/1376] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:36:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:36:29.202982 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ee58a8-c96a-4ea6-b82c-c9696b61c037 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:29.203809 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ee58a8-c96a-4ea6-b82c-c9696b61c037 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:29.203971 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ee58a8-c96a-4ea6-b82c-c9696b61c037 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:29.207478 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 696/1377] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:36:29 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:36:29.215392 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db633c7e-e1d5-46df-8c75-251feccbeb17 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:29.215854 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db633c7e-e1d5-46df-8c75-251feccbeb17 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:29.215967 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db633c7e-e1d5-46df-8c75-251feccbeb17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:29.238952 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-db633c7e-e1d5-46df-8c75-251feccbeb17 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:36:29.240149 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-db633c7e-e1d5-46df-8c75-251feccbeb17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:36:29.248699 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 682/1378] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:36:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:36:29.305261 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ee58a8-c96a-4ea6-b82c-c9696b61c037 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:29.320988 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ee58a8-c96a-4ea6-b82c-c9696b61c037 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:29.326569 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33a40f3-6635-48d9-8b37-6e96164b4a81 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f86c16179a70474a817be9fda33ece9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:29.327215 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33a40f3-6635-48d9-8b37-6e96164b4a81 service nova] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:29.327502 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33a40f3-6635-48d9-8b37-6e96164b4a81 service nova] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:29.327782 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33a40f3-6635-48d9-8b37-6e96164b4a81 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:29.337702 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33a40f3-6635-48d9-8b37-6e96164b4a81 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:29.339468 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33a40f3-6635-48d9-8b37-6e96164b4a81 service nova] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:29.350019 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 697/1379] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:36:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:36:31.607642 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f5a7ae-67c8-40d5-8cc2-ec117122f021 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:31.608190 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f5a7ae-67c8-40d5-8cc2-ec117122f021 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:31.608190 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f5a7ae-67c8-40d5-8cc2-ec117122f021 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:31.609624 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 683/1380] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:36:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:36:31.615652 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3978faff-e20a-4d8d-8ded-77e5fa06dfa1 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:31.616104 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3978faff-e20a-4d8d-8ded-77e5fa06dfa1 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:31.616520 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3978faff-e20a-4d8d-8ded-77e5fa06dfa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:31.641621 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-3978faff-e20a-4d8d-8ded-77e5fa06dfa1 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:36:31.642820 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-3978faff-e20a-4d8d-8ded-77e5fa06dfa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:36:31.653919 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 698/1381] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:36:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:36:31.699229 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f5a7ae-67c8-40d5-8cc2-ec117122f021 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:31.714321 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f5a7ae-67c8-40d5-8cc2-ec117122f021 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:31.720418 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309e9827-d61a-4111-877f-f59409eae6fd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f86c16179a70474a817be9fda33ece9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:31.721068 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309e9827-d61a-4111-877f-f59409eae6fd service nova] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:31.721351 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309e9827-d61a-4111-877f-f59409eae6fd service nova] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:31.721625 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309e9827-d61a-4111-877f-f59409eae6fd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:31.731361 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309e9827-d61a-4111-877f-f59409eae6fd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:31.732976 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309e9827-d61a-4111-877f-f59409eae6fd service nova] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:31.745093 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 684/1382] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:36:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:36:33.940550 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66d7c7f-39e7-44f6-bcb2-803670ffde72 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:33.941032 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66d7c7f-39e7-44f6-bcb2-803670ffde72 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:33.941127 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66d7c7f-39e7-44f6-bcb2-803670ffde72 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:33.942464 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 699/1383] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:36:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:36:33.948022 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c89b3d8-8238-4fa6-85f1-2c0112dcefab None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:33.948464 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c89b3d8-8238-4fa6-85f1-2c0112dcefab None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:33.948628 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c89b3d8-8238-4fa6-85f1-2c0112dcefab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:33.967027 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-3c89b3d8-8238-4fa6-85f1-2c0112dcefab None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:36:33.968223 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-3c89b3d8-8238-4fa6-85f1-2c0112dcefab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:36:33.978752 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 685/1384] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:36:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:36:34.027898 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66d7c7f-39e7-44f6-bcb2-803670ffde72 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:34.043841 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66d7c7f-39e7-44f6-bcb2-803670ffde72 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:34.049312 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d923202d-e903-4f7e-840c-4d83c4a36fef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f86c16179a70474a817be9fda33ece9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:34.049954 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d923202d-e903-4f7e-840c-4d83c4a36fef service nova] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:34.050240 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d923202d-e903-4f7e-840c-4d83c4a36fef service nova] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:34.050520 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d923202d-e903-4f7e-840c-4d83c4a36fef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:34.059904 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d923202d-e903-4f7e-840c-4d83c4a36fef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:34.061590 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d923202d-e903-4f7e-840c-4d83c4a36fef service nova] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:34.072722 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 700/1385] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:36:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:36:36.290412 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99aab58-883e-4686-a230-d3e1fac7f317 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:36.290958 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99aab58-883e-4686-a230-d3e1fac7f317 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:36.290958 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99aab58-883e-4686-a230-d3e1fac7f317 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:36.292387 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 686/1386] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:36:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:36:36.298618 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8ed2d5-e63e-4af4-947d-b1a19bd25323 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:36.299164 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8ed2d5-e63e-4af4-947d-b1a19bd25323 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:36.299164 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8ed2d5-e63e-4af4-947d-b1a19bd25323 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:36.316962 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-aa8ed2d5-e63e-4af4-947d-b1a19bd25323 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:36:36.318122 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-aa8ed2d5-e63e-4af4-947d-b1a19bd25323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:36:36.327637 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 701/1387] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:36:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:36:36.383791 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99aab58-883e-4686-a230-d3e1fac7f317 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:36.399219 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99aab58-883e-4686-a230-d3e1fac7f317 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:36.404721 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08ebc5a-7f20-454e-b30e-5772aa5a1e79 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f86c16179a70474a817be9fda33ece9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:36.405381 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08ebc5a-7f20-454e-b30e-5772aa5a1e79 service nova] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:36.405672 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08ebc5a-7f20-454e-b30e-5772aa5a1e79 service nova] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:36.405941 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08ebc5a-7f20-454e-b30e-5772aa5a1e79 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:36.414686 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08ebc5a-7f20-454e-b30e-5772aa5a1e79 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:36.416291 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08ebc5a-7f20-454e-b30e-5772aa5a1e79 service nova] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:36.426170 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 687/1388] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:36:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 17:36:38.570487 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749c62a6-5e76-41f0-a757-9fcbbf144fd1 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:38.571113 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749c62a6-5e76-41f0-a757-9fcbbf144fd1 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:38.571113 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749c62a6-5e76-41f0-a757-9fcbbf144fd1 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:38.572560 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 702/1389] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:36:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:36:38.578256 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3678bab-59dc-49e6-b4fc-dd59ace5a5d2 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:38.578671 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3678bab-59dc-49e6-b4fc-dd59ace5a5d2 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:38.578827 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3678bab-59dc-49e6-b4fc-dd59ace5a5d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:38.598849 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-c3678bab-59dc-49e6-b4fc-dd59ace5a5d2 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:36:38.600084 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-c3678bab-59dc-49e6-b4fc-dd59ace5a5d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:36:38.609221 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 688/1390] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:36:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:36:38.655706 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749c62a6-5e76-41f0-a757-9fcbbf144fd1 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:38.670915 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749c62a6-5e76-41f0-a757-9fcbbf144fd1 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:38.676639 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73e9022-8d32-4beb-9f88-07c54cfefa28 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f86c16179a70474a817be9fda33ece9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:38.677304 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73e9022-8d32-4beb-9f88-07c54cfefa28 service nova] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:38.677581 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73e9022-8d32-4beb-9f88-07c54cfefa28 service nova] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:38.677867 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73e9022-8d32-4beb-9f88-07c54cfefa28 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:38.686759 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73e9022-8d32-4beb-9f88-07c54cfefa28 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:38.688413 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73e9022-8d32-4beb-9f88-07c54cfefa28 service nova] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:38.699019 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 703/1391] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:36:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 17:36:40.914634 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6f9dde-a508-4367-85c0-a8f30fd5b849 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:40.915182 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6f9dde-a508-4367-85c0-a8f30fd5b849 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:40.915182 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6f9dde-a508-4367-85c0-a8f30fd5b849 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:40.916635 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 689/1392] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:36:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:36:40.922065 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e99a87e-8a45-4ee0-8e45-eb8ce17a3987 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:40.922488 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e99a87e-8a45-4ee0-8e45-eb8ce17a3987 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:40.922635 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e99a87e-8a45-4ee0-8e45-eb8ce17a3987 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:40.940877 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-8e99a87e-8a45-4ee0-8e45-eb8ce17a3987 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:36:40.942087 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-8e99a87e-8a45-4ee0-8e45-eb8ce17a3987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:36:40.950463 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 704/1393] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:36:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:36:40.995947 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6f9dde-a508-4367-85c0-a8f30fd5b849 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:41.010994 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6f9dde-a508-4367-85c0-a8f30fd5b849 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:41.016649 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e5b57d-e793-4566-bdda-b574d5baad20 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f86c16179a70474a817be9fda33ece9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:41.017286 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e5b57d-e793-4566-bdda-b574d5baad20 service nova] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:41.017573 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e5b57d-e793-4566-bdda-b574d5baad20 service nova] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:41.017851 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e5b57d-e793-4566-bdda-b574d5baad20 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:41.027103 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e5b57d-e793-4566-bdda-b574d5baad20 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:41.028800 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e5b57d-e793-4566-bdda-b574d5baad20 service nova] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:41.039702 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 690/1394] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:36:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:36:43.245172 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c79261-7efc-402e-8251-23879fc14c81 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:43.245769 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c79261-7efc-402e-8251-23879fc14c81 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:43.245769 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c79261-7efc-402e-8251-23879fc14c81 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:43.247161 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 705/1395] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:36:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:36:43.252566 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bb3bee-2c78-4bc8-9a94-6b92c0f334ff None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:43.252982 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bb3bee-2c78-4bc8-9a94-6b92c0f334ff None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:43.253245 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bb3bee-2c78-4bc8-9a94-6b92c0f334ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:43.271721 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-b4bb3bee-2c78-4bc8-9a94-6b92c0f334ff None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:36:43.273548 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-b4bb3bee-2c78-4bc8-9a94-6b92c0f334ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:36:43.281463 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 691/1396] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:36:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:36:43.325431 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c79261-7efc-402e-8251-23879fc14c81 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:43.339954 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c79261-7efc-402e-8251-23879fc14c81 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:43.345434 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e8e7de-7b6e-4803-bf10-367356749131 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f86c16179a70474a817be9fda33ece9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:43.346077 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e8e7de-7b6e-4803-bf10-367356749131 service nova] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:43.346363 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e8e7de-7b6e-4803-bf10-367356749131 service nova] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:43.346644 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e8e7de-7b6e-4803-bf10-367356749131 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:43.355251 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e8e7de-7b6e-4803-bf10-367356749131 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:43.356869 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e8e7de-7b6e-4803-bf10-367356749131 service nova] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:43.366939 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 706/1397] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:36:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:36:43.713196 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062244e7-d523-4dd7-a405-25f301da22c2 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:43.713486 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062244e7-d523-4dd7-a405-25f301da22c2 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:43.713767 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062244e7-d523-4dd7-a405-25f301da22c2 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:43.715160 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 692/1398] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:36:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:36:43.720418 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8030dbd8-5474-4465-a440-16570f206862 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:43.720780 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8030dbd8-5474-4465-a440-16570f206862 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:43.721060 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8030dbd8-5474-4465-a440-16570f206862 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:43.738845 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-8030dbd8-5474-4465-a440-16570f206862 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:36:43.740056 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-8030dbd8-5474-4465-a440-16570f206862 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:36:43.748274 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 707/1399] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:36:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:36:43.805203 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72912d75-65d2-424c-aea5-3f838bcdf1b1 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:43.805488 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72912d75-65d2-424c-aea5-3f838bcdf1b1 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:43.805764 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72912d75-65d2-424c-aea5-3f838bcdf1b1 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:43.807196 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 693/1400] 10.222.0.42 () {54 vars in 962 bytes} [Thu Jun 16 17:36:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:36:43.812277 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d51bb6-8ec0-45a5-9eb9-b45a06cfeb6f None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:43.812597 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d51bb6-8ec0-45a5-9eb9-b45a06cfeb6f None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:43.812901 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d51bb6-8ec0-45a5-9eb9-b45a06cfeb6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:43.861459 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-97d51bb6-8ec0-45a5-9eb9-b45a06cfeb6f None None] MFA Rules not processed for user `d89029c1b3134df1962c041988bead74`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:36:43.862706 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-97d51bb6-8ec0-45a5-9eb9-b45a06cfeb6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:36:43.901671 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:36:43.930615 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 708/1401] 10.222.0.42 () {58 vars in 1059 bytes} [Thu Jun 16 17:36:43 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:36:43.938292 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72912d75-65d2-424c-aea5-3f838bcdf1b1 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:43.943601 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=55 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:36:43.951238 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72912d75-65d2-424c-aea5-3f838bcdf1b1 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:43.956475 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf80d0e-a575-4be0-829b-e108ed2a91bc service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd89029c1b3134df1962c041988bead74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:43.957488 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf80d0e-a575-4be0-829b-e108ed2a91bc service glance] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:43.958004 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf80d0e-a575-4be0-829b-e108ed2a91bc service glance] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:43.958527 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf80d0e-a575-4be0-829b-e108ed2a91bc service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:43.971570 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf80d0e-a575-4be0-829b-e108ed2a91bc service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:43.973160 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf80d0e-a575-4be0-829b-e108ed2a91bc service glance] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:43.983685 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 694/1402] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:36:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:36:44.745718 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d0c8a7-605b-4d16-84e0-9bcaa4a9faa9 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:44.746180 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d0c8a7-605b-4d16-84e0-9bcaa4a9faa9 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:44.746513 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d0c8a7-605b-4d16-84e0-9bcaa4a9faa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:44.796520 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-43d0c8a7-605b-4d16-84e0-9bcaa4a9faa9 None None] MFA Rules not processed for user `71443c30b4554719b5549493bf3cc00c`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:36:44.797748 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-43d0c8a7-605b-4d16-84e0-9bcaa4a9faa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:36:44.863669 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 709/1403] 10.222.0.42 () {58 vars in 1009 bytes} [Thu Jun 16 17:36:44 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:36:44.910111 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5682629-26a9-4fab-a4c2-b6db9792aca2 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:44.910551 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5682629-26a9-4fab-a4c2-b6db9792aca2 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:44.910655 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5682629-26a9-4fab-a4c2-b6db9792aca2 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:44.912029 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 695/1404] 10.222.0.42 () {54 vars in 942 bytes} [Thu Jun 16 17:36:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:36:44.919246 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345fa572-31f2-41bf-83be-9cd6020ae061 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:44.919536 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345fa572-31f2-41bf-83be-9cd6020ae061 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:44.919809 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345fa572-31f2-41bf-83be-9cd6020ae061 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:44.977897 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-345fa572-31f2-41bf-83be-9cd6020ae061 None None] MFA Rules not processed for user `47b7a9cd615c41d6983a0007b475038c`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:36:44.979144 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-345fa572-31f2-41bf-83be-9cd6020ae061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:36:45.052123 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 710/1405] 10.222.0.42 () {58 vars in 1039 bytes} [Thu Jun 16 17:36:44 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:36:45.062073 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5682629-26a9-4fab-a4c2-b6db9792aca2 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:45.075837 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5682629-26a9-4fab-a4c2-b6db9792aca2 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:45.080754 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedcd2c9-202f-401d-9f07-eaa75541ba14 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47b7a9cd615c41d6983a0007b475038c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:45.081387 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedcd2c9-202f-401d-9f07-eaa75541ba14 service swift] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:45.081669 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedcd2c9-202f-401d-9f07-eaa75541ba14 service swift] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:45.081940 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedcd2c9-202f-401d-9f07-eaa75541ba14 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:45.090392 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedcd2c9-202f-401d-9f07-eaa75541ba14 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:45.091995 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedcd2c9-202f-401d-9f07-eaa75541ba14 service swift] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:45.100077 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 696/1406] 10.222.0.42 () {60 vars in 1384 bytes} [Thu Jun 16 17:36:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 17:36:45.620751 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f848868-08ba-477c-9dd2-c9b87fbff9fa None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:45.621029 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f848868-08ba-477c-9dd2-c9b87fbff9fa None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:45.621313 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f848868-08ba-477c-9dd2-c9b87fbff9fa None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:45.622826 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 711/1407] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:36:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:36:45.627844 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90c1878-1cd2-415b-8422-3675b12cc15c None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:45.628329 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90c1878-1cd2-415b-8422-3675b12cc15c None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:45.628782 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90c1878-1cd2-415b-8422-3675b12cc15c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:45.649934 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-f90c1878-1cd2-415b-8422-3675b12cc15c None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:36:45.651141 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-f90c1878-1cd2-415b-8422-3675b12cc15c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:36:45.659534 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 697/1408] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:36:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:36:45.706603 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f848868-08ba-477c-9dd2-c9b87fbff9fa None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:45.727205 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f848868-08ba-477c-9dd2-c9b87fbff9fa None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:45.734617 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f593f6-5fbc-4c96-a44c-fa533f2eb0c1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f86c16179a70474a817be9fda33ece9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:45.735350 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f593f6-5fbc-4c96-a44c-fa533f2eb0c1 service nova] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:45.735706 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f593f6-5fbc-4c96-a44c-fa533f2eb0c1 service nova] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:45.736057 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f593f6-5fbc-4c96-a44c-fa533f2eb0c1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:45.751096 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f593f6-5fbc-4c96-a44c-fa533f2eb0c1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:45.752850 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f593f6-5fbc-4c96-a44c-fa533f2eb0c1 service nova] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:45.758241 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:36:45.763750 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 712/1409] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:36:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:36:55.641719 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=14 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:37:08.164361 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e994d1-6441-4bca-a466-1afc9ef92b91 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:37:08.165091 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e994d1-6441-4bca-a466-1afc9ef92b91 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:37:08.165091 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e994d1-6441-4bca-a466-1afc9ef92b91 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:37:08.166399 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 698/1410] 10.222.0.42 () {54 vars in 918 bytes} [Thu Jun 16 17:37:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:37:08.171549 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701e0c09-74d2-4dd6-ba6a-6ca1d1ab72d5 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:37:08.172008 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701e0c09-74d2-4dd6-ba6a-6ca1d1ab72d5 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:37:08.172111 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701e0c09-74d2-4dd6-ba6a-6ca1d1ab72d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:37:08.189877 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-701e0c09-74d2-4dd6-ba6a-6ca1d1ab72d5 None None] MFA Rules not processed for user `0570d8d9508d4546ae897276f7d181d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:37:08.191064 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-701e0c09-74d2-4dd6-ba6a-6ca1d1ab72d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:37:08.199830 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 713/1411] 10.222.0.42 () {58 vars in 1015 bytes} [Thu Jun 16 17:37:08 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:37:08.214571 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbf545a-b036-4051-86ef-0a73c4d667eb None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:37:08.214887 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbf545a-b036-4051-86ef-0a73c4d667eb None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:37:08.215168 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbf545a-b036-4051-86ef-0a73c4d667eb None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:37:08.216562 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 699/1412] 10.222.0.42 () {54 vars in 918 bytes} [Thu Jun 16 17:37:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:37:08.221095 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df69372f-c51d-475f-b65c-ff755b73d170 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:37:08.221382 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df69372f-c51d-475f-b65c-ff755b73d170 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:37:08.221662 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df69372f-c51d-475f-b65c-ff755b73d170 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:37:08.240186 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-df69372f-c51d-475f-b65c-ff755b73d170 None None] MFA Rules not processed for user `f86c16179a70474a817be9fda33ece9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:37:08.241398 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-df69372f-c51d-475f-b65c-ff755b73d170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:37:08.250275 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 714/1413] 10.222.0.42 () {58 vars in 1015 bytes} [Thu Jun 16 17:37:08 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:37:10.880787 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b5f5de-d2f9-4165-b081-6b160fe58b30 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:37:10.881353 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b5f5de-d2f9-4165-b081-6b160fe58b30 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:37:10.881353 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b5f5de-d2f9-4165-b081-6b160fe58b30 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:37:10.882759 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 700/1414] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 17:37:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:37:10.888221 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ecbba7-339f-4f10-b5fc-a45e638baf19 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:37:10.888629 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ecbba7-339f-4f10-b5fc-a45e638baf19 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:37:10.888812 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ecbba7-339f-4f10-b5fc-a45e638baf19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:37:10.907185 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-e7ecbba7-339f-4f10-b5fc-a45e638baf19 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:37:10.908382 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-e7ecbba7-339f-4f10-b5fc-a45e638baf19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:37:10.916861 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 715/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 17:37:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:37:10.961427 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b5f5de-d2f9-4165-b081-6b160fe58b30 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:37:10.977334 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b5f5de-d2f9-4165-b081-6b160fe58b30 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:37:10.982900 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d24046-c94d-40ed-b893-71ec6f2b938f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f86c16179a70474a817be9fda33ece9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:37:10.983539 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d24046-c94d-40ed-b893-71ec6f2b938f service nova] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:37:10.983825 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d24046-c94d-40ed-b893-71ec6f2b938f service nova] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:37:10.984103 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d24046-c94d-40ed-b893-71ec6f2b938f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:37:10.993248 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d24046-c94d-40ed-b893-71ec6f2b938f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:37:10.994838 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d24046-c94d-40ed-b893-71ec6f2b938f service nova] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:37:11.004477 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 701/1416] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:37:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:37:11.978040 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0b2b83-ecbe-43c0-bc23-aeaf33fb44a6 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:37:11.978580 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0b2b83-ecbe-43c0-bc23-aeaf33fb44a6 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:37:11.978580 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0b2b83-ecbe-43c0-bc23-aeaf33fb44a6 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:37:11.979947 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 716/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 17:37:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:37:11.984800 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac964b02-95ed-4f74-b358-af3d21987ae1 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:37:11.985264 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac964b02-95ed-4f74-b358-af3d21987ae1 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:37:11.985362 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac964b02-95ed-4f74-b358-af3d21987ae1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:37:12.003504 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-ac964b02-95ed-4f74-b358-af3d21987ae1 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:37:12.005480 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-ac964b02-95ed-4f74-b358-af3d21987ae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:37:12.013366 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 702/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 17:37:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:37:12.179716 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0b2b83-ecbe-43c0-bc23-aeaf33fb44a6 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:37:12.195018 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0b2b83-ecbe-43c0-bc23-aeaf33fb44a6 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:37:12.200713 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a620807-0141-42e2-a2d2-dc24db91f4cd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f86c16179a70474a817be9fda33ece9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:37:12.201355 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a620807-0141-42e2-a2d2-dc24db91f4cd service nova] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:37:12.201640 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a620807-0141-42e2-a2d2-dc24db91f4cd service nova] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:37:12.201935 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a620807-0141-42e2-a2d2-dc24db91f4cd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:37:12.210347 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a620807-0141-42e2-a2d2-dc24db91f4cd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:37:12.211923 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a620807-0141-42e2-a2d2-dc24db91f4cd service nova] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:37:12.221407 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 717/1419] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:37:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:37:13.157295 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b482521d-da16-48db-8d69-4c5dc099c038 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:37:13.157809 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b482521d-da16-48db-8d69-4c5dc099c038 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:37:13.157809 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b482521d-da16-48db-8d69-4c5dc099c038 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:37:13.159177 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 703/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 17:37:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:37:13.165101 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e9472c-9bdb-4ba9-9540-dd9710f71baf None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:37:13.165544 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e9472c-9bdb-4ba9-9540-dd9710f71baf None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:37:13.165667 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e9472c-9bdb-4ba9-9540-dd9710f71baf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:37:13.184228 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-91e9472c-9bdb-4ba9-9540-dd9710f71baf None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:37:13.185462 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-91e9472c-9bdb-4ba9-9540-dd9710f71baf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:37:13.193708 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 718/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 17:37:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:37:13.346159 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b482521d-da16-48db-8d69-4c5dc099c038 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:37:13.361011 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b482521d-da16-48db-8d69-4c5dc099c038 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:37:13.366491 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf51573-8eab-447f-b686-882a358b4d29 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f86c16179a70474a817be9fda33ece9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:37:13.367123 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf51573-8eab-447f-b686-882a358b4d29 service nova] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:37:13.367403 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf51573-8eab-447f-b686-882a358b4d29 service nova] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:37:13.367675 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf51573-8eab-447f-b686-882a358b4d29 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:37:13.375675 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf51573-8eab-447f-b686-882a358b4d29 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:37:13.377267 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf51573-8eab-447f-b686-882a358b4d29 service nova] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:37:13.386666 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 704/1422] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:37:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:37:14.338920 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcf1499-ea0d-460c-ba86-48e672e1f077 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:37:14.339500 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcf1499-ea0d-460c-ba86-48e672e1f077 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:37:14.339500 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcf1499-ea0d-460c-ba86-48e672e1f077 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:37:14.340978 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 719/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 17:37:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:37:14.346079 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7690d755-905f-461d-b8fa-9c3eb1f1c5fb None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:37:14.346466 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7690d755-905f-461d-b8fa-9c3eb1f1c5fb None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:37:14.346634 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7690d755-905f-461d-b8fa-9c3eb1f1c5fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:37:14.365449 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-7690d755-905f-461d-b8fa-9c3eb1f1c5fb None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:37:14.366650 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-7690d755-905f-461d-b8fa-9c3eb1f1c5fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:37:14.374904 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 705/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 17:37:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:37:14.530800 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcf1499-ea0d-460c-ba86-48e672e1f077 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:37:14.546546 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcf1499-ea0d-460c-ba86-48e672e1f077 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:37:14.552908 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd8b20e-bec1-452d-b7c4-9119ee375a8e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f86c16179a70474a817be9fda33ece9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:37:14.553574 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd8b20e-bec1-452d-b7c4-9119ee375a8e service nova] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:37:14.553907 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd8b20e-bec1-452d-b7c4-9119ee375a8e service nova] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:37:14.554193 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd8b20e-bec1-452d-b7c4-9119ee375a8e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:37:14.562762 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd8b20e-bec1-452d-b7c4-9119ee375a8e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:37:14.564327 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd8b20e-bec1-452d-b7c4-9119ee375a8e service nova] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:37:14.574371 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 720/1425] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:37:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:37:15.491522 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8a21da-450e-40c5-88c0-13510bb3e391 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:37:15.492090 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8a21da-450e-40c5-88c0-13510bb3e391 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:37:15.492090 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8a21da-450e-40c5-88c0-13510bb3e391 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:37:15.493535 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 706/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 17:37:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:37:15.498481 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585e6b48-f5a2-4448-97b8-dee25a8fd57b None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:37:15.498982 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585e6b48-f5a2-4448-97b8-dee25a8fd57b None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:37:15.499155 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585e6b48-f5a2-4448-97b8-dee25a8fd57b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:37:15.517924 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-585e6b48-f5a2-4448-97b8-dee25a8fd57b None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:37:15.519849 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-585e6b48-f5a2-4448-97b8-dee25a8fd57b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:37:15.528418 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 721/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 17:37:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:37:15.682458 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8a21da-450e-40c5-88c0-13510bb3e391 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:37:15.696836 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8a21da-450e-40c5-88c0-13510bb3e391 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:37:15.702323 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b3881c-2467-47b6-bf75-4029c7c57861 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f86c16179a70474a817be9fda33ece9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:37:15.702965 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b3881c-2467-47b6-bf75-4029c7c57861 service nova] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:37:15.703267 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b3881c-2467-47b6-bf75-4029c7c57861 service nova] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:37:15.703546 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b3881c-2467-47b6-bf75-4029c7c57861 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:37:15.712926 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b3881c-2467-47b6-bf75-4029c7c57861 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:37:15.714502 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b3881c-2467-47b6-bf75-4029c7c57861 service nova] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:37:15.724397 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 707/1428] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:37:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:37:16.598154 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b269db20-dbba-4643-aefc-3d8d24ce2472 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:37:16.598728 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b269db20-dbba-4643-aefc-3d8d24ce2472 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:37:16.598728 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b269db20-dbba-4643-aefc-3d8d24ce2472 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:37:16.600071 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 722/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 17:37:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:37:16.605155 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb14e353-6d84-4dcb-bb16-596c7dc5ea63 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:37:16.605576 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb14e353-6d84-4dcb-bb16-596c7dc5ea63 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:37:16.605714 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb14e353-6d84-4dcb-bb16-596c7dc5ea63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:37:16.624123 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-fb14e353-6d84-4dcb-bb16-596c7dc5ea63 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:37:16.625362 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-fb14e353-6d84-4dcb-bb16-596c7dc5ea63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:37:16.634264 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 708/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 17:37:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:37:16.787147 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b269db20-dbba-4643-aefc-3d8d24ce2472 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:37:16.802167 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b269db20-dbba-4643-aefc-3d8d24ce2472 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:37:16.807547 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5166a49-7edb-4855-b5a8-55f6d500dd95 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f86c16179a70474a817be9fda33ece9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:37:16.808184 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5166a49-7edb-4855-b5a8-55f6d500dd95 service nova] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:37:16.808503 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5166a49-7edb-4855-b5a8-55f6d500dd95 service nova] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:37:16.808774 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5166a49-7edb-4855-b5a8-55f6d500dd95 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:37:16.817275 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5166a49-7edb-4855-b5a8-55f6d500dd95 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:37:16.818885 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5166a49-7edb-4855-b5a8-55f6d500dd95 service nova] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:37:16.828293 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 723/1431] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:37:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:37:17.696509 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75300d5c-3f9f-4462-b2c4-9db7fd761ec2 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:37:17.697053 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75300d5c-3f9f-4462-b2c4-9db7fd761ec2 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:37:17.697053 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75300d5c-3f9f-4462-b2c4-9db7fd761ec2 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:37:17.698470 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 709/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 17:37:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:37:17.703559 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eedbc8-c195-4e59-a703-85e09867c990 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:37:17.704477 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eedbc8-c195-4e59-a703-85e09867c990 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:37:17.704764 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eedbc8-c195-4e59-a703-85e09867c990 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:37:17.730000 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-93eedbc8-c195-4e59-a703-85e09867c990 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:37:17.730449 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-93eedbc8-c195-4e59-a703-85e09867c990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:37:17.742476 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 724/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 17:37:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:37:17.905556 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75300d5c-3f9f-4462-b2c4-9db7fd761ec2 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:37:17.919936 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75300d5c-3f9f-4462-b2c4-9db7fd761ec2 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:37:17.926182 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a0bc4e-1cbe-4a27-b1ca-9b5a6c3cbc29 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f86c16179a70474a817be9fda33ece9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:37:17.926879 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a0bc4e-1cbe-4a27-b1ca-9b5a6c3cbc29 service nova] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:37:17.927200 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a0bc4e-1cbe-4a27-b1ca-9b5a6c3cbc29 service nova] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:37:17.927479 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a0bc4e-1cbe-4a27-b1ca-9b5a6c3cbc29 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:37:17.935960 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a0bc4e-1cbe-4a27-b1ca-9b5a6c3cbc29 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:37:17.937571 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a0bc4e-1cbe-4a27-b1ca-9b5a6c3cbc29 service nova] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:37:17.948000 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 710/1434] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:37:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 17:37:18.819177 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b051d68d-c207-4630-ab3c-7cb5eab51376 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:37:18.819905 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b051d68d-c207-4630-ab3c-7cb5eab51376 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:37:18.819905 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b051d68d-c207-4630-ab3c-7cb5eab51376 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:37:18.821162 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 725/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 17:37:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:37:18.826377 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d49650-0189-4908-8153-1e1390af904c None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:37:18.826797 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d49650-0189-4908-8153-1e1390af904c None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:37:18.826932 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d49650-0189-4908-8153-1e1390af904c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:37:18.844871 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-e0d49650-0189-4908-8153-1e1390af904c None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:37:18.846105 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-e0d49650-0189-4908-8153-1e1390af904c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:37:18.853985 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 711/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 17:37:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:37:19.012217 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b051d68d-c207-4630-ab3c-7cb5eab51376 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:37:19.027672 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b051d68d-c207-4630-ab3c-7cb5eab51376 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:37:19.033147 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704d7620-242d-4b52-a528-dbe3792ea4d9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f86c16179a70474a817be9fda33ece9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:37:19.033793 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704d7620-242d-4b52-a528-dbe3792ea4d9 service nova] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:37:19.034077 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704d7620-242d-4b52-a528-dbe3792ea4d9 service nova] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:37:19.034892 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704d7620-242d-4b52-a528-dbe3792ea4d9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:37:19.043255 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704d7620-242d-4b52-a528-dbe3792ea4d9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:37:19.044872 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704d7620-242d-4b52-a528-dbe3792ea4d9 service nova] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:37:19.054999 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 726/1437] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:37:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:37:19.912124 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25f5d78-09a1-4cfe-8eba-17f0335e7370 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:37:19.912658 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25f5d78-09a1-4cfe-8eba-17f0335e7370 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:37:19.912756 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25f5d78-09a1-4cfe-8eba-17f0335e7370 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:37:19.914113 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 712/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 17:37:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:37:19.919029 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa33642-3f08-4848-8877-cbe95fa8360b None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:37:19.919471 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa33642-3f08-4848-8877-cbe95fa8360b None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:37:19.919582 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa33642-3f08-4848-8877-cbe95fa8360b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:37:19.937433 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-2fa33642-3f08-4848-8877-cbe95fa8360b None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:37:19.938638 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-2fa33642-3f08-4848-8877-cbe95fa8360b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:37:19.947468 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 727/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 17:37:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:37:20.101943 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25f5d78-09a1-4cfe-8eba-17f0335e7370 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:37:20.120672 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25f5d78-09a1-4cfe-8eba-17f0335e7370 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:37:20.131211 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32081872-e001-42ef-9c64-391a3f5fd808 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f86c16179a70474a817be9fda33ece9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:37:20.131882 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32081872-e001-42ef-9c64-391a3f5fd808 service nova] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:37:20.132168 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32081872-e001-42ef-9c64-391a3f5fd808 service nova] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:37:20.132500 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32081872-e001-42ef-9c64-391a3f5fd808 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:37:20.142031 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32081872-e001-42ef-9c64-391a3f5fd808 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:37:20.143553 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32081872-e001-42ef-9c64-391a3f5fd808 service nova] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:37:20.153763 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 713/1440] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:37:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:37:22.212524 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6e4633-e028-4946-8606-236a39124128 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:37:22.213086 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6e4633-e028-4946-8606-236a39124128 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:37:22.213086 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6e4633-e028-4946-8606-236a39124128 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:37:22.214492 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 728/1441] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 17:37:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:37:22.219569 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a99a70-c53c-4b5c-bad7-bd2f643c9fe2 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:37:22.219998 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a99a70-c53c-4b5c-bad7-bd2f643c9fe2 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:37:22.220125 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a99a70-c53c-4b5c-bad7-bd2f643c9fe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:37:22.239486 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-25a99a70-c53c-4b5c-bad7-bd2f643c9fe2 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:37:22.243356 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-25a99a70-c53c-4b5c-bad7-bd2f643c9fe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:37:22.248910 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 714/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 17:37:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:37:22.288397 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6e4633-e028-4946-8606-236a39124128 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:37:22.299091 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6e4633-e028-4946-8606-236a39124128 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:37:22.302770 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a68d89-47d0-4a1f-b02d-d8f69f8aa22b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:37:22.303406 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a68d89-47d0-4a1f-b02d-d8f69f8aa22b service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:37:22.303691 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a68d89-47d0-4a1f-b02d-d8f69f8aa22b service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:37:22.303976 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a68d89-47d0-4a1f-b02d-d8f69f8aa22b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:37:22.312905 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a68d89-47d0-4a1f-b02d-d8f69f8aa22b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:37:22.314328 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a68d89-47d0-4a1f-b02d-d8f69f8aa22b service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:37:22.323941 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 729/1443] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:37:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:37:24.624707 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34aced7-bd87-413c-9a54-305612675ac3 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:37:24.625246 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34aced7-bd87-413c-9a54-305612675ac3 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:37:24.625372 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34aced7-bd87-413c-9a54-305612675ac3 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:37:24.626727 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 715/1444] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 17:37:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:37:24.632741 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0b2708-bb78-4f02-a944-4ed207279827 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:37:24.633146 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0b2708-bb78-4f02-a944-4ed207279827 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:37:24.633334 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0b2708-bb78-4f02-a944-4ed207279827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:37:24.653980 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-8a0b2708-bb78-4f02-a944-4ed207279827 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:37:24.655176 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-8a0b2708-bb78-4f02-a944-4ed207279827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:37:24.664638 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 730/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 17:37:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:37:24.703271 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34aced7-bd87-413c-9a54-305612675ac3 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:37:24.704957 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-d34aced7-bd87-413c-9a54-305612675ac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:37:24.718326 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34aced7-bd87-413c-9a54-305612675ac3 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:37:24.722610 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fca2da-8a45-4175-a79a-caec4ca6306f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:37:24.723181 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fca2da-8a45-4175-a79a-caec4ca6306f service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:37:24.723455 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fca2da-8a45-4175-a79a-caec4ca6306f service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:37:24.723723 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fca2da-8a45-4175-a79a-caec4ca6306f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:37:24.732587 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fca2da-8a45-4175-a79a-caec4ca6306f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:37:24.734025 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fca2da-8a45-4175-a79a-caec4ca6306f service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:37:24.743510 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 716/1446] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:37:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:37:27.211161 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0857e8d4-b8cb-4beb-81f9-d406d26ccddd None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:37:27.211734 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0857e8d4-b8cb-4beb-81f9-d406d26ccddd None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:37:27.211734 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0857e8d4-b8cb-4beb-81f9-d406d26ccddd None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:37:27.213133 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 731/1447] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 17:37:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:37:27.218476 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9bd212-451a-4168-88c7-89831cc6f646 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:37:27.218904 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9bd212-451a-4168-88c7-89831cc6f646 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:37:27.219035 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9bd212-451a-4168-88c7-89831cc6f646 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:37:27.236795 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-ea9bd212-451a-4168-88c7-89831cc6f646 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:37:27.238242 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-ea9bd212-451a-4168-88c7-89831cc6f646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:37:27.246095 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 717/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 17:37:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:37:27.285578 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0857e8d4-b8cb-4beb-81f9-d406d26ccddd None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:37:27.296088 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0857e8d4-b8cb-4beb-81f9-d406d26ccddd None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:37:27.299630 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cebd7a2-de5e-4cb9-a43b-68faf5450103 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:37:27.300327 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cebd7a2-de5e-4cb9-a43b-68faf5450103 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:37:27.300647 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cebd7a2-de5e-4cb9-a43b-68faf5450103 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:37:27.300925 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cebd7a2-de5e-4cb9-a43b-68faf5450103 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:37:27.309736 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cebd7a2-de5e-4cb9-a43b-68faf5450103 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:37:27.311155 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cebd7a2-de5e-4cb9-a43b-68faf5450103 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:37:27.321226 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 732/1449] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:37:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:37:29.585132 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c452b0-e825-4e58-b60f-469971b8063f None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:37:29.585694 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c452b0-e825-4e58-b60f-469971b8063f None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:37:29.585694 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c452b0-e825-4e58-b60f-469971b8063f None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:37:29.586990 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 718/1450] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 17:37:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:37:29.592075 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817dbdcb-d20e-471d-a17b-dc666683f825 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:37:29.592553 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817dbdcb-d20e-471d-a17b-dc666683f825 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:37:29.592663 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817dbdcb-d20e-471d-a17b-dc666683f825 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:37:29.612211 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-817dbdcb-d20e-471d-a17b-dc666683f825 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:37:29.613592 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-817dbdcb-d20e-471d-a17b-dc666683f825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:37:29.621434 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 733/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 17:37:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:37:29.660178 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c452b0-e825-4e58-b60f-469971b8063f None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:37:29.671115 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c452b0-e825-4e58-b60f-469971b8063f None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:37:29.676505 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d567d12b-54af-4db7-a04c-f2ae90666688 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:37:29.677498 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d567d12b-54af-4db7-a04c-f2ae90666688 service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:37:29.677790 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d567d12b-54af-4db7-a04c-f2ae90666688 service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:37:29.678064 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d567d12b-54af-4db7-a04c-f2ae90666688 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:37:29.687282 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d567d12b-54af-4db7-a04c-f2ae90666688 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:37:29.688748 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d567d12b-54af-4db7-a04c-f2ae90666688 service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:37:29.698405 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 719/1452] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:37:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:37:32.037510 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465843e4-ba2f-4711-9e01-0ed1069490bf None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:37:32.038048 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465843e4-ba2f-4711-9e01-0ed1069490bf None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:37:32.038048 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465843e4-ba2f-4711-9e01-0ed1069490bf None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:37:32.039358 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 734/1453] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 17:37:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:37:32.044696 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86bf687-cfe8-4aad-aaa8-37ba01d84cf9 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:37:32.045121 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86bf687-cfe8-4aad-aaa8-37ba01d84cf9 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:37:32.045289 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86bf687-cfe8-4aad-aaa8-37ba01d84cf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:37:32.062620 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-f86bf687-cfe8-4aad-aaa8-37ba01d84cf9 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:37:32.063820 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-f86bf687-cfe8-4aad-aaa8-37ba01d84cf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:37:32.071711 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 720/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 17:37:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:37:32.110558 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465843e4-ba2f-4711-9e01-0ed1069490bf None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:37:32.121291 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465843e4-ba2f-4711-9e01-0ed1069490bf None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:37:32.124876 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a14df62-3740-45b9-8677-81706e7517ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:37:32.125515 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a14df62-3740-45b9-8677-81706e7517ff service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:37:32.125797 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a14df62-3740-45b9-8677-81706e7517ff service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:37:32.126077 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a14df62-3740-45b9-8677-81706e7517ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:37:32.135515 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a14df62-3740-45b9-8677-81706e7517ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:37:32.136990 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a14df62-3740-45b9-8677-81706e7517ff service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:37:32.147279 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 735/1455] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:37:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:37:34.412602 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e91db4-bdc8-4478-9fd7-da561f106146 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:37:34.413189 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e91db4-bdc8-4478-9fd7-da561f106146 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:37:34.413189 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e91db4-bdc8-4478-9fd7-da561f106146 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:37:34.414592 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 721/1456] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 17:37:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:37:34.419709 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47ada7f-2c8e-4069-9699-2149db5a69f6 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:37:34.420154 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47ada7f-2c8e-4069-9699-2149db5a69f6 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:37:34.420262 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47ada7f-2c8e-4069-9699-2149db5a69f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:37:34.438097 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-d47ada7f-2c8e-4069-9699-2149db5a69f6 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:37:34.439310 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-d47ada7f-2c8e-4069-9699-2149db5a69f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:37:34.447618 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 736/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 17:37:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:37:34.486577 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e91db4-bdc8-4478-9fd7-da561f106146 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:37:34.497247 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e91db4-bdc8-4478-9fd7-da561f106146 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:37:34.501270 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a549650a-1871-4856-9d55-635f6b068d76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:37:34.501896 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a549650a-1871-4856-9d55-635f6b068d76 service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:37:34.502205 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a549650a-1871-4856-9d55-635f6b068d76 service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:37:34.502436 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a549650a-1871-4856-9d55-635f6b068d76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:37:34.511275 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a549650a-1871-4856-9d55-635f6b068d76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:37:34.512726 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a549650a-1871-4856-9d55-635f6b068d76 service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:37:34.522769 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 722/1458] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:37:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:37:36.776877 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598f7ac3-3adb-4630-8444-788379e57e51 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:37:36.777549 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598f7ac3-3adb-4630-8444-788379e57e51 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:37:36.777549 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598f7ac3-3adb-4630-8444-788379e57e51 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:37:36.778798 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 737/1459] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 17:37:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:37:36.786605 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a977eba7-71b6-43cf-b86f-8b9bda701c3c None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:37:36.787035 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a977eba7-71b6-43cf-b86f-8b9bda701c3c None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:37:36.787145 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a977eba7-71b6-43cf-b86f-8b9bda701c3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:37:36.793175 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=70 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:37:36.805795 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-a977eba7-71b6-43cf-b86f-8b9bda701c3c None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:37:36.806989 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-a977eba7-71b6-43cf-b86f-8b9bda701c3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:37:36.814853 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 723/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 17:37:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:37:36.852743 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598f7ac3-3adb-4630-8444-788379e57e51 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:37:36.858081 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=73 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:37:36.864243 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598f7ac3-3adb-4630-8444-788379e57e51 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:37:36.868235 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cf37f1-8b8e-49c6-8104-04c988f71bff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:37:36.868916 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cf37f1-8b8e-49c6-8104-04c988f71bff service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:37:36.869199 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cf37f1-8b8e-49c6-8104-04c988f71bff service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:37:36.869474 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cf37f1-8b8e-49c6-8104-04c988f71bff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:37:36.879431 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cf37f1-8b8e-49c6-8104-04c988f71bff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:37:36.880906 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cf37f1-8b8e-49c6-8104-04c988f71bff service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:37:36.890995 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 738/1461] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:37:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:37:39.074010 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca59345-1187-4219-b9da-6692a57210e8 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:37:39.074598 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca59345-1187-4219-b9da-6692a57210e8 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:37:39.074598 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca59345-1187-4219-b9da-6692a57210e8 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:37:39.076014 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 724/1462] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 17:37:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:37:39.081945 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb5e4d9-a76b-4f33-9bf8-3eec8e6f2b4e None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:37:39.082450 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb5e4d9-a76b-4f33-9bf8-3eec8e6f2b4e None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:37:39.082561 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb5e4d9-a76b-4f33-9bf8-3eec8e6f2b4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:37:39.103363 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-ddb5e4d9-a76b-4f33-9bf8-3eec8e6f2b4e None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:37:39.104609 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-ddb5e4d9-a76b-4f33-9bf8-3eec8e6f2b4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:37:39.113376 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 739/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 17:37:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:37:39.151296 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca59345-1187-4219-b9da-6692a57210e8 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:37:39.162545 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca59345-1187-4219-b9da-6692a57210e8 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:37:39.166062 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4e21f5-75f1-4b73-af23-52fd74eb7bdb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:37:39.166695 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4e21f5-75f1-4b73-af23-52fd74eb7bdb service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:37:39.166979 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4e21f5-75f1-4b73-af23-52fd74eb7bdb service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:37:39.167252 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4e21f5-75f1-4b73-af23-52fd74eb7bdb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:37:39.177002 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4e21f5-75f1-4b73-af23-52fd74eb7bdb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:37:39.178419 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4e21f5-75f1-4b73-af23-52fd74eb7bdb service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:37:39.187760 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 725/1464] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:37:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:37:41.351990 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788b0064-f8d3-44e6-b70c-3b11ad080aa2 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:37:41.352523 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788b0064-f8d3-44e6-b70c-3b11ad080aa2 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:37:41.352624 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788b0064-f8d3-44e6-b70c-3b11ad080aa2 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:37:41.353972 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 740/1465] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 17:37:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:37:41.359081 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b32650-ea1c-48dd-9a2e-fdda0b7647f1 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:37:41.359465 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b32650-ea1c-48dd-9a2e-fdda0b7647f1 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:37:41.359633 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b32650-ea1c-48dd-9a2e-fdda0b7647f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:37:41.378162 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-b7b32650-ea1c-48dd-9a2e-fdda0b7647f1 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:37:41.379427 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-b7b32650-ea1c-48dd-9a2e-fdda0b7647f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:37:41.387234 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 726/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 17:37:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:37:41.424840 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788b0064-f8d3-44e6-b70c-3b11ad080aa2 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:37:41.435411 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788b0064-f8d3-44e6-b70c-3b11ad080aa2 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:37:41.439515 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b59e498-fe62-4264-82ef-32a2c6d50e22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:37:41.440153 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b59e498-fe62-4264-82ef-32a2c6d50e22 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:37:41.440472 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b59e498-fe62-4264-82ef-32a2c6d50e22 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:37:41.440751 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b59e498-fe62-4264-82ef-32a2c6d50e22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:37:41.449646 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b59e498-fe62-4264-82ef-32a2c6d50e22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:37:41.451065 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b59e498-fe62-4264-82ef-32a2c6d50e22 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:37:41.461210 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 741/1467] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:37:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:37:43.664869 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94d3599-89bd-4006-89fd-e2cd72bd21ee None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:37:43.665425 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94d3599-89bd-4006-89fd-e2cd72bd21ee None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:37:43.665536 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94d3599-89bd-4006-89fd-e2cd72bd21ee None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:37:43.667251 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 727/1468] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 17:37:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:37:43.671995 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dca2f00-fe3d-49e8-afae-2edc5f2f09b9 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:37:43.672408 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dca2f00-fe3d-49e8-afae-2edc5f2f09b9 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:37:43.672587 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dca2f00-fe3d-49e8-afae-2edc5f2f09b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:37:43.691909 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-9dca2f00-fe3d-49e8-afae-2edc5f2f09b9 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:37:43.693148 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-9dca2f00-fe3d-49e8-afae-2edc5f2f09b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:37:43.701219 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 742/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 17:37:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:37:43.739021 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94d3599-89bd-4006-89fd-e2cd72bd21ee None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:37:43.750057 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94d3599-89bd-4006-89fd-e2cd72bd21ee None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:37:43.753898 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837ae91f-f9da-4b73-830d-9b353887b102 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:37:43.754536 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837ae91f-f9da-4b73-830d-9b353887b102 service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:37:43.754821 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837ae91f-f9da-4b73-830d-9b353887b102 service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:37:43.755097 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837ae91f-f9da-4b73-830d-9b353887b102 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:37:43.764752 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837ae91f-f9da-4b73-830d-9b353887b102 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:37:43.766434 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837ae91f-f9da-4b73-830d-9b353887b102 service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:37:43.776678 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 728/1470] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:37:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:37:53.685905 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=17 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:37:53.772039 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=17 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:38:04.178652 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1087a0-25a9-489f-a23e-3385331de353 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:38:04.179251 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1087a0-25a9-489f-a23e-3385331de353 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:38:04.179251 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1087a0-25a9-489f-a23e-3385331de353 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:38:04.180695 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 743/1471] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:38:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:38:04.185859 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5d528c-a160-4867-a6ae-d6d08c4125bf None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:38:04.186271 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5d528c-a160-4867-a6ae-d6d08c4125bf None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:38:04.186450 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5d528c-a160-4867-a6ae-d6d08c4125bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:38:04.204032 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-6e5d528c-a160-4867-a6ae-d6d08c4125bf None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:38:04.205296 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-6e5d528c-a160-4867-a6ae-d6d08c4125bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:38:04.213140 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 729/1472] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:38:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:38:04.260718 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57717a6-788b-4915-990b-9d4cbf205ee3 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:38:04.261009 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57717a6-788b-4915-990b-9d4cbf205ee3 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:38:04.261296 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57717a6-788b-4915-990b-9d4cbf205ee3 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:38:04.262647 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 744/1473] 10.222.0.42 () {54 vars in 962 bytes} [Thu Jun 16 17:38:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:38:04.268095 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eab4b1-436a-4cef-8418-d21bcb6b656f None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:38:04.268512 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eab4b1-436a-4cef-8418-d21bcb6b656f None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:38:04.268803 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eab4b1-436a-4cef-8418-d21bcb6b656f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:38:04.287080 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-11eab4b1-436a-4cef-8418-d21bcb6b656f None None] MFA Rules not processed for user `d89029c1b3134df1962c041988bead74`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:38:04.288284 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-11eab4b1-436a-4cef-8418-d21bcb6b656f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:38:04.294700 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 730/1474] 10.222.0.42 () {58 vars in 1059 bytes} [Thu Jun 16 17:38:04 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:38:04.301931 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57717a6-788b-4915-990b-9d4cbf205ee3 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:38:04.313294 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57717a6-788b-4915-990b-9d4cbf205ee3 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:38:04.316816 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21eb8f56-359b-4f3b-9ec1-aa547ea5252b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd89029c1b3134df1962c041988bead74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:38:04.317448 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eb8f56-359b-4f3b-9ec1-aa547ea5252b service glance] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:38:04.317755 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eb8f56-359b-4f3b-9ec1-aa547ea5252b service glance] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:38:04.318033 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eb8f56-359b-4f3b-9ec1-aa547ea5252b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:38:04.326993 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21eb8f56-359b-4f3b-9ec1-aa547ea5252b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:38:04.328994 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21eb8f56-359b-4f3b-9ec1-aa547ea5252b service glance] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:38:04.338326 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 745/1475] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:38:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:38:06.582718 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2257f9-7989-42a0-9356-8a4e7f3fb904 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:38:06.583267 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2257f9-7989-42a0-9356-8a4e7f3fb904 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:38:06.583267 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2257f9-7989-42a0-9356-8a4e7f3fb904 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:38:06.584680 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 731/1476] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:38:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:38:06.589807 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d501964-15d9-428d-a929-14cae264be2e None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:38:06.590223 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d501964-15d9-428d-a929-14cae264be2e None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:38:06.590356 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d501964-15d9-428d-a929-14cae264be2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:38:06.609153 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-6d501964-15d9-428d-a929-14cae264be2e None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:38:06.610367 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-6d501964-15d9-428d-a929-14cae264be2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:38:06.618315 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 746/1477] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:38:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:38:06.642488 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2257f9-7989-42a0-9356-8a4e7f3fb904 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:38:06.654865 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2257f9-7989-42a0-9356-8a4e7f3fb904 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:38:06.659842 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3f646f-1876-408a-9d10-5c24ed79357c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:38:06.660527 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3f646f-1876-408a-9d10-5c24ed79357c admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:38:06.660812 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3f646f-1876-408a-9d10-5c24ed79357c admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:38:06.661084 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3f646f-1876-408a-9d10-5c24ed79357c admin admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:38:06.661832 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3f646f-1876-408a-9d10-5c24ed79357c admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:38:06.663639 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: WARNING py.warnings [None req-0c3f646f-1876-408a-9d10-5c24ed79357c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 17:38:06.663639 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: warnings.warn(msg) Jun 16 17:38:06.663639 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: Jun 16 17:38:06.664820 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3f646f-1876-408a-9d10-5c24ed79357c admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:38:06.683579 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 732/1478] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jun 16 17:38:06 2022] GET /identity/v3/projects => generated 2553 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:38:08.842051 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400719d5-d43f-4294-9e5e-37f3b3084650 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:38:08.842682 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400719d5-d43f-4294-9e5e-37f3b3084650 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:38:08.842978 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400719d5-d43f-4294-9e5e-37f3b3084650 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:38:08.844568 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 747/1479] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:38:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:38:08.849680 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2929e13e-ca9e-4e58-97f0-d1c01d8ed635 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:38:08.850128 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2929e13e-ca9e-4e58-97f0-d1c01d8ed635 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:38:08.850277 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2929e13e-ca9e-4e58-97f0-d1c01d8ed635 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:38:08.869324 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-2929e13e-ca9e-4e58-97f0-d1c01d8ed635 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:38:08.870529 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-2929e13e-ca9e-4e58-97f0-d1c01d8ed635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:38:08.879462 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 733/1480] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:38:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:38:08.923470 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400719d5-d43f-4294-9e5e-37f3b3084650 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:38:08.937965 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400719d5-d43f-4294-9e5e-37f3b3084650 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:38:08.943405 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a46940-0b8c-4ace-8b1d-255a5d325088 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f86c16179a70474a817be9fda33ece9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:38:08.944042 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a46940-0b8c-4ace-8b1d-255a5d325088 service nova] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:38:08.944328 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a46940-0b8c-4ace-8b1d-255a5d325088 service nova] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:38:08.944645 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a46940-0b8c-4ace-8b1d-255a5d325088 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:38:08.953391 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a46940-0b8c-4ace-8b1d-255a5d325088 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:38:08.954962 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a46940-0b8c-4ace-8b1d-255a5d325088 service nova] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:38:08.964590 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 748/1481] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:38:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:38:11.335273 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e7cddf-2124-4ab2-afdd-40ecdf323278 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:38:11.335847 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e7cddf-2124-4ab2-afdd-40ecdf323278 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:38:11.335847 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e7cddf-2124-4ab2-afdd-40ecdf323278 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:38:11.337287 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 734/1482] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:38:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:38:11.342686 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c513831c-5bed-4ccd-ba74-56744b246ce8 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:38:11.343094 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c513831c-5bed-4ccd-ba74-56744b246ce8 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:38:11.343455 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c513831c-5bed-4ccd-ba74-56744b246ce8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:38:11.361561 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-c513831c-5bed-4ccd-ba74-56744b246ce8 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:38:11.362739 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-c513831c-5bed-4ccd-ba74-56744b246ce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:38:11.375926 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 749/1483] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:38:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:38:11.413280 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e7cddf-2124-4ab2-afdd-40ecdf323278 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:38:11.424606 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e7cddf-2124-4ab2-afdd-40ecdf323278 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:38:11.428094 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc970f84-b4d2-4e1b-b651-e4233721b9a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:38:11.428761 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc970f84-b4d2-4e1b-b651-e4233721b9a6 service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:38:11.429050 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc970f84-b4d2-4e1b-b651-e4233721b9a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:38:11.429329 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc970f84-b4d2-4e1b-b651-e4233721b9a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:38:11.438262 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc970f84-b4d2-4e1b-b651-e4233721b9a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:38:11.439687 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc970f84-b4d2-4e1b-b651-e4233721b9a6 service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:38:11.449172 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 735/1484] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:38:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:38:13.638178 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e26863a-56e7-4518-b5d5-4a2c34862be1 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:38:13.639503 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e26863a-56e7-4518-b5d5-4a2c34862be1 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:38:13.639503 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e26863a-56e7-4518-b5d5-4a2c34862be1 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:38:13.641396 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 750/1485] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:38:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:38:13.646642 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa26b7e-7f0c-4321-a096-b7cf40f7595d None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:38:13.647092 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa26b7e-7f0c-4321-a096-b7cf40f7595d None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:38:13.647199 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa26b7e-7f0c-4321-a096-b7cf40f7595d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:38:13.665717 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-aaa26b7e-7f0c-4321-a096-b7cf40f7595d None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:38:13.666907 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-aaa26b7e-7f0c-4321-a096-b7cf40f7595d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:38:13.675387 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 736/1486] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:38:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:38:13.713602 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e26863a-56e7-4518-b5d5-4a2c34862be1 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:38:13.724329 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e26863a-56e7-4518-b5d5-4a2c34862be1 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:38:13.727834 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bd2aa2-b58d-465f-916e-1a7287ee2c20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:38:13.728492 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bd2aa2-b58d-465f-916e-1a7287ee2c20 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:38:13.728774 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bd2aa2-b58d-465f-916e-1a7287ee2c20 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:38:13.729046 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bd2aa2-b58d-465f-916e-1a7287ee2c20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:38:13.738608 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bd2aa2-b58d-465f-916e-1a7287ee2c20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:38:13.740017 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bd2aa2-b58d-465f-916e-1a7287ee2c20 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:38:13.749899 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 751/1487] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:38:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:38:16.003526 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e2830b-b9c6-46a0-8c14-698980384cdf None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:38:16.004085 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e2830b-b9c6-46a0-8c14-698980384cdf None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:38:16.004209 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e2830b-b9c6-46a0-8c14-698980384cdf None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:38:16.005567 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 737/1488] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:38:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:38:16.010659 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa87cdd-1bb2-4f0a-8ffb-ce5affa55bf8 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:38:16.011066 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa87cdd-1bb2-4f0a-8ffb-ce5affa55bf8 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:38:16.011225 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa87cdd-1bb2-4f0a-8ffb-ce5affa55bf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:38:16.029951 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-caa87cdd-1bb2-4f0a-8ffb-ce5affa55bf8 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:38:16.031155 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-caa87cdd-1bb2-4f0a-8ffb-ce5affa55bf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:38:16.039080 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 752/1489] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:38:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:38:16.079381 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e2830b-b9c6-46a0-8c14-698980384cdf None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:38:16.089943 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e2830b-b9c6-46a0-8c14-698980384cdf None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:38:16.093607 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cefcc18-9ed4-49c2-af7f-ee5a8e419ade service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:38:16.094240 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cefcc18-9ed4-49c2-af7f-ee5a8e419ade service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:38:16.094527 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cefcc18-9ed4-49c2-af7f-ee5a8e419ade service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:38:16.094804 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cefcc18-9ed4-49c2-af7f-ee5a8e419ade service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:38:16.103561 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cefcc18-9ed4-49c2-af7f-ee5a8e419ade service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:38:16.105043 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cefcc18-9ed4-49c2-af7f-ee5a8e419ade service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:38:16.114839 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 738/1490] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:38:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:38:18.463698 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315e3cca-6685-4047-8cdd-9daf6499a76e None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:38:18.464298 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315e3cca-6685-4047-8cdd-9daf6499a76e None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:38:18.464298 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315e3cca-6685-4047-8cdd-9daf6499a76e None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:38:18.465665 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 753/1491] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jun 16 17:38:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:38:18.471065 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07154431-a9a0-417a-b609-370b8993e24b None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:38:18.471498 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07154431-a9a0-417a-b609-370b8993e24b None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:38:18.471634 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07154431-a9a0-417a-b609-370b8993e24b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:38:18.489748 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-07154431-a9a0-417a-b609-370b8993e24b None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:38:18.490948 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-07154431-a9a0-417a-b609-370b8993e24b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:38:18.499756 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 739/1492] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jun 16 17:38:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:38:18.538983 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315e3cca-6685-4047-8cdd-9daf6499a76e None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:38:18.550577 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315e3cca-6685-4047-8cdd-9daf6499a76e None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:38:18.554155 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ea5d1a-313b-4eb1-92dd-691c726e4496 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:38:18.554792 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ea5d1a-313b-4eb1-92dd-691c726e4496 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:38:18.555074 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ea5d1a-313b-4eb1-92dd-691c726e4496 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:38:18.555380 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ea5d1a-313b-4eb1-92dd-691c726e4496 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:38:18.564399 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ea5d1a-313b-4eb1-92dd-691c726e4496 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:38:18.565842 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ea5d1a-313b-4eb1-92dd-691c726e4496 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:38:18.575164 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 754/1493] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:38:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:38:28.484595 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=33 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:38:28.570961 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=33 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:39:55.675441 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6f0dca-c51e-46b9-bbe9-29b919411a7c None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:39:55.676016 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6f0dca-c51e-46b9-bbe9-29b919411a7c None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:39:55.676016 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6f0dca-c51e-46b9-bbe9-29b919411a7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:39:55.694715 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-2b6f0dca-c51e-46b9-bbe9-29b919411a7c None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:39:55.696128 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-2b6f0dca-c51e-46b9-bbe9-29b919411a7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:39:55.704287 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 740/1494] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:39:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:39:55.713704 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d1f3e9-988b-4e80-a957-af167048791d None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:39:55.714235 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d1f3e9-988b-4e80-a957-af167048791d None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:39:55.714235 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d1f3e9-988b-4e80-a957-af167048791d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:39:55.732729 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-e6d1f3e9-988b-4e80-a957-af167048791d None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:39:55.734365 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-e6d1f3e9-988b-4e80-a957-af167048791d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:39:55.743005 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 755/1495] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:39:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:39:55.747441 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6f0dca-c51e-46b9-bbe9-29b919411a7c None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:39:55.756967 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6f0dca-c51e-46b9-bbe9-29b919411a7c None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:39:55.762195 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb36e4f-559d-4d16-975b-f23c5400b0f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:39:55.762840 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb36e4f-559d-4d16-975b-f23c5400b0f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:39:55.763120 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb36e4f-559d-4d16-975b-f23c5400b0f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:39:55.763392 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb36e4f-559d-4d16-975b-f23c5400b0f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:39:55.764108 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb36e4f-559d-4d16-975b-f23c5400b0f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:39:55.764440 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb36e4f-559d-4d16-975b-f23c5400b0f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:39:55.765449 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: WARNING py.warnings [None req-fcb36e4f-559d-4d16-975b-f23c5400b0f0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 16 17:39:55.765449 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: warnings.warn(msg) Jun 16 17:39:55.765449 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: Jun 16 17:39:55.766319 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb36e4f-559d-4d16-975b-f23c5400b0f0 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:39:55.778483 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 741/1496] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jun 16 17:39:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:39:55.786511 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d1f3e9-988b-4e80-a957-af167048791d None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:39:55.796273 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d1f3e9-988b-4e80-a957-af167048791d None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:39:55.800894 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb38dab8-d0ff-499b-811d-8895e2ce952f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:39:55.801549 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb38dab8-d0ff-499b-811d-8895e2ce952f admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:39:55.801834 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb38dab8-d0ff-499b-811d-8895e2ce952f admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:39:55.802114 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb38dab8-d0ff-499b-811d-8895e2ce952f admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:39:55.802796 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb38dab8-d0ff-499b-811d-8895e2ce952f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:39:55.804732 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: WARNING py.warnings [None req-bb38dab8-d0ff-499b-811d-8895e2ce952f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 17:39:55.804732 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: warnings.warn(msg) Jun 16 17:39:55.804732 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: Jun 16 17:39:55.806092 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb38dab8-d0ff-499b-811d-8895e2ce952f admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:39:55.843215 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 756/1497] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jun 16 17:39:55 2022] POST /identity/v3/projects => generated 379 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:39:55.847476 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb36e4f-559d-4d16-975b-f23c5400b0f0 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:39:55.859492 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb36e4f-559d-4d16-975b-f23c5400b0f0 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:39:55.918717 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dca1c9d-1070-49a6-bfcb-140a96a9c89a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:39:55.919394 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dca1c9d-1070-49a6-bfcb-140a96a9c89a admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:39:55.919682 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dca1c9d-1070-49a6-bfcb-140a96a9c89a admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:39:55.919960 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dca1c9d-1070-49a6-bfcb-140a96a9c89a admin admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:39:55.920671 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dca1c9d-1070-49a6-bfcb-140a96a9c89a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:39:55.921738 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: WARNING py.warnings [None req-4dca1c9d-1070-49a6-bfcb-140a96a9c89a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 17:39:55.921738 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: warnings.warn(msg) Jun 16 17:39:55.921738 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: Jun 16 17:39:55.922926 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dca1c9d-1070-49a6-bfcb-140a96a9c89a admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:39:55.942751 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 742/1498] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jun 16 17:39:55 2022] POST /identity/v3/users => generated 290 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:39:55.947466 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb38dab8-d0ff-499b-811d-8895e2ce952f admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:39:55.957720 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb38dab8-d0ff-499b-811d-8895e2ce952f admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:39:55.963217 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e1d721-c551-43dd-a9cf-0d85acd49383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:39:55.963873 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e1d721-c551-43dd-a9cf-0d85acd49383 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:39:55.964156 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e1d721-c551-43dd-a9cf-0d85acd49383 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:39:55.964460 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e1d721-c551-43dd-a9cf-0d85acd49383 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:39:55.965177 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e1d721-c551-43dd-a9cf-0d85acd49383 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:39:55.966190 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: WARNING py.warnings [None req-e0e1d721-c551-43dd-a9cf-0d85acd49383 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 16 17:39:55.966190 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: warnings.warn(msg) Jun 16 17:39:55.966190 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: Jun 16 17:39:55.967059 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e1d721-c551-43dd-a9cf-0d85acd49383 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:39:55.980907 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 757/1499] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:39:55 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:39:55.985062 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dca1c9d-1070-49a6-bfcb-140a96a9c89a admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:39:55.997612 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dca1c9d-1070-49a6-bfcb-140a96a9c89a admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:39:56.004500 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8acc6f-268d-46ac-ad1a-2b962415b960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:39:56.005170 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8acc6f-268d-46ac-ad1a-2b962415b960 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:39:56.005457 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8acc6f-268d-46ac-ad1a-2b962415b960 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:39:56.005736 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8acc6f-268d-46ac-ad1a-2b962415b960 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:39:56.006460 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8acc6f-268d-46ac-ad1a-2b962415b960 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:39:56.007475 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: WARNING py.warnings [None req-6a8acc6f-268d-46ac-ad1a-2b962415b960 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 16 17:39:56.007475 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: warnings.warn(msg) Jun 16 17:39:56.007475 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: Jun 16 17:39:56.008362 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8acc6f-268d-46ac-ad1a-2b962415b960 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:39:56.022927 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 743/1500] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:39:55 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:39:56.027776 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e1d721-c551-43dd-a9cf-0d85acd49383 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:39:56.037995 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e1d721-c551-43dd-a9cf-0d85acd49383 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:39:56.045438 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc440ba-366c-4158-ae08-1266a63032be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:39:56.046315 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc440ba-366c-4158-ae08-1266a63032be admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:39:56.046599 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc440ba-366c-4158-ae08-1266a63032be admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:39:56.046910 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc440ba-366c-4158-ae08-1266a63032be admin admin] PATH_INFO: `/v3/projects/8c6a2a956fdc4e30aa954a5b6f8a015f/users/5cb4895ecc9746a18bb0624f630fd8ba/roles/0be2075946f54c8485f1c275f746478a` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:39:56.064321 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc440ba-366c-4158-ae08-1266a63032be admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c6a2a956fdc4e30aa954a5b6f8a015f, user_id=5cb4895ecc9746a18bb0624f630fd8ba, role_id=0be2075946f54c8485f1c275f746478a)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:39:56.065381 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: WARNING py.warnings [None req-bdc440ba-366c-4158-ae08-1266a63032be admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 17:39:56.065381 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: warnings.warn(msg) Jun 16 17:39:56.065381 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: Jun 16 17:39:56.067298 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc440ba-366c-4158-ae08-1266a63032be admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:39:56.080677 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 758/1501] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:39:56 2022] PUT /identity/v3/projects/8c6a2a956fdc4e30aa954a5b6f8a015f/users/5cb4895ecc9746a18bb0624f630fd8ba/roles/0be2075946f54c8485f1c275f746478a => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:39:56.092909 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8fb0dd-6bfa-434f-86d8-c18b93dfc1e4 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:39:56.093203 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8fb0dd-6bfa-434f-86d8-c18b93dfc1e4 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:39:56.093504 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8fb0dd-6bfa-434f-86d8-c18b93dfc1e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:39:56.120687 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-9d8fb0dd-6bfa-434f-86d8-c18b93dfc1e4 None None] MFA Rules not processed for user `5cb4895ecc9746a18bb0624f630fd8ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:39:56.121908 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-9d8fb0dd-6bfa-434f-86d8-c18b93dfc1e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:39:56.185806 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 744/1502] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:39:56 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:39:56.199393 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b901e40-2fba-4926-a8f2-616b6e8482fb None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:39:56.199628 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b901e40-2fba-4926-a8f2-616b6e8482fb None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:39:56.199912 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b901e40-2fba-4926-a8f2-616b6e8482fb None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:39:56.201319 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 759/1503] 10.222.0.42 () {54 vars in 963 bytes} [Thu Jun 16 17:39:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:39:56.206113 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbca529a-fa53-4df8-ae74-eef69f35b2f2 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:39:56.206403 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbca529a-fa53-4df8-ae74-eef69f35b2f2 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:39:56.206682 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbca529a-fa53-4df8-ae74-eef69f35b2f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:39:56.225406 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-dbca529a-fa53-4df8-ae74-eef69f35b2f2 None None] MFA Rules not processed for user `24d5e2637b7043e08313909d5b06e329`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:39:56.227597 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-dbca529a-fa53-4df8-ae74-eef69f35b2f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:39:56.271046 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:39:56.280065 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 745/1504] 10.222.0.42 () {58 vars in 1060 bytes} [Thu Jun 16 17:39:56 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:39:56.287654 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b901e40-2fba-4926-a8f2-616b6e8482fb None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:39:56.299231 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b901e40-2fba-4926-a8f2-616b6e8482fb None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:39:56.302930 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9412e061-8457-42e3-ac0d-fc4b753cd9ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24d5e2637b7043e08313909d5b06e329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:39:56.303563 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9412e061-8457-42e3-ac0d-fc4b753cd9ef service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:39:56.303848 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9412e061-8457-42e3-ac0d-fc4b753cd9ef service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:39:56.304124 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9412e061-8457-42e3-ac0d-fc4b753cd9ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:39:56.313486 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9412e061-8457-42e3-ac0d-fc4b753cd9ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:39:56.314914 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9412e061-8457-42e3-ac0d-fc4b753cd9ef service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:39:56.323686 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 760/1505] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:39:56 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 17:39:56.365820 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d312522-47cb-418d-b580-a462eefc21c7 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:39:56.366115 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d312522-47cb-418d-b580-a462eefc21c7 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:39:56.366396 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d312522-47cb-418d-b580-a462eefc21c7 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:39:56.368416 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 746/1506] 10.222.0.42 () {52 vars in 829 bytes} [Thu Jun 16 17:39:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:39:56.372161 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9412e061-8457-42e3-ac0d-fc4b753cd9ef service cinder] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:39:56.434256 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9412e061-8457-42e3-ac0d-fc4b753cd9ef service cinder] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:39:56.438759 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db69da8-0357-42b4-b417-d4a7e63ec726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:39:56.439616 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db69da8-0357-42b4-b417-d4a7e63ec726 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:39:56.439892 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db69da8-0357-42b4-b417-d4a7e63ec726 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:39:56.440161 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db69da8-0357-42b4-b417-d4a7e63ec726 admin admin] PATH_INFO: `/v3/users/5cb4895ecc9746a18bb0624f630fd8ba` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:39:56.441353 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db69da8-0357-42b4-b417-d4a7e63ec726 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5cb4895ecc9746a18bb0624f630fd8ba)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:39:56.442390 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: WARNING py.warnings [None req-1db69da8-0357-42b4-b417-d4a7e63ec726 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 17:39:56.442390 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: warnings.warn(msg) Jun 16 17:39:56.442390 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: Jun 16 17:39:56.443577 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db69da8-0357-42b4-b417-d4a7e63ec726 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:39:56.511004 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-1db69da8-0357-42b4-b417-d4a7e63ec726 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5cb4895ecc9746a18bb0624f630fd8ba'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:39:56.529645 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-1db69da8-0357-42b4-b417-d4a7e63ec726 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5cb4895ecc9746a18bb0624f630fd8ba'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:39:56.530321 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-1db69da8-0357-42b4-b417-d4a7e63ec726 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5cb4895ecc9746a18bb0624f630fd8ba'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:39:56.542996 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-1db69da8-0357-42b4-b417-d4a7e63ec726 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5cb4895ecc9746a18bb0624f630fd8ba'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:39:56.546936 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:39:56.552904 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-1db69da8-0357-42b4-b417-d4a7e63ec726 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5cb4895ecc9746a18bb0624f630fd8ba'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:39:56.560428 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 761/1507] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jun 16 17:39:56 2022] DELETE /identity/v3/users/5cb4895ecc9746a18bb0624f630fd8ba => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:39:56.564973 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aac3d7-84c2-41f4-b419-54cf62b83aa4 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:39:56.565255 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aac3d7-84c2-41f4-b419-54cf62b83aa4 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:39:56.565524 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aac3d7-84c2-41f4-b419-54cf62b83aa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:39:56.582319 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-51aac3d7-84c2-41f4-b419-54cf62b83aa4 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:39:56.583496 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-51aac3d7-84c2-41f4-b419-54cf62b83aa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:39:56.640941 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 747/1508] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:39:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:39:56.651052 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db69da8-0357-42b4-b417-d4a7e63ec726 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:39:56.652548 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-1db69da8-0357-42b4-b417-d4a7e63ec726 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:39:56.709964 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db69da8-0357-42b4-b417-d4a7e63ec726 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:39:56.714994 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0e0a59-c81f-4fba-89ec-3fa697716102 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:39:56.715637 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0e0a59-c81f-4fba-89ec-3fa697716102 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:39:56.715921 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0e0a59-c81f-4fba-89ec-3fa697716102 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:39:56.716196 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0e0a59-c81f-4fba-89ec-3fa697716102 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:39:56.725159 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0e0a59-c81f-4fba-89ec-3fa697716102 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:39:56.727705 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0e0a59-c81f-4fba-89ec-3fa697716102 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:39:56.737216 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 762/1509] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:39:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:39:57.093864 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51aac3d7-84c2-41f4-b419-54cf62b83aa4 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:39:57.095308 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-51aac3d7-84c2-41f4-b419-54cf62b83aa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:39:57.107491 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51aac3d7-84c2-41f4-b419-54cf62b83aa4 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:39:57.112460 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6c2220-009a-48e7-99ff-ac1a5fe37b0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:39:57.114394 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6c2220-009a-48e7-99ff-ac1a5fe37b0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:39:57.114680 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6c2220-009a-48e7-99ff-ac1a5fe37b0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:39:57.115086 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6c2220-009a-48e7-99ff-ac1a5fe37b0f admin admin] PATH_INFO: `/v3/projects/8c6a2a956fdc4e30aa954a5b6f8a015f` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:39:57.116243 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6c2220-009a-48e7-99ff-ac1a5fe37b0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c6a2a956fdc4e30aa954a5b6f8a015f)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:39:57.117485 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: WARNING py.warnings [None req-4e6c2220-009a-48e7-99ff-ac1a5fe37b0f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 17:39:57.117485 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: warnings.warn(msg) Jun 16 17:39:57.117485 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: Jun 16 17:39:57.118707 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6c2220-009a-48e7-99ff-ac1a5fe37b0f admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:39:57.183449 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-4e6c2220-009a-48e7-99ff-ac1a5fe37b0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c6a2a956fdc4e30aa954a5b6f8a015f', 'request_id': 'req-4e6c2220-009a-48e7-99ff-ac1a5fe37b0f'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:39:57.193569 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-4e6c2220-009a-48e7-99ff-ac1a5fe37b0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c6a2a956fdc4e30aa954a5b6f8a015f', 'request_id': 'req-4e6c2220-009a-48e7-99ff-ac1a5fe37b0f'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:39:57.199862 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-4e6c2220-009a-48e7-99ff-ac1a5fe37b0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c6a2a956fdc4e30aa954a5b6f8a015f', 'request_id': 'req-4e6c2220-009a-48e7-99ff-ac1a5fe37b0f'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:39:57.218171 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-4e6c2220-009a-48e7-99ff-ac1a5fe37b0f admin admin] The token cache is being invalidate because project 8c6a2a956fdc4e30aa954a5b6f8a015f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80357) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:39:57.218463 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-4e6c2220-009a-48e7-99ff-ac1a5fe37b0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:39:57.219117 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-4e6c2220-009a-48e7-99ff-ac1a5fe37b0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:39:57.220826 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 748/1510] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jun 16 17:39:57 2022] DELETE /identity/v3/projects/8c6a2a956fdc4e30aa954a5b6f8a015f => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:40:06.733017 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:40:07.215216 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:41:45.203651 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df5dceb-c9aa-49fc-a93c-0cef9ec4f411 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:45.204231 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df5dceb-c9aa-49fc-a93c-0cef9ec4f411 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:45.204231 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df5dceb-c9aa-49fc-a93c-0cef9ec4f411 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:45.206360 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 763/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 17:41:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:41:45.211342 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3143c6-d04a-49f5-b138-38e02e5b3fa6 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:45.211770 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3143c6-d04a-49f5-b138-38e02e5b3fa6 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:45.211903 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3143c6-d04a-49f5-b138-38e02e5b3fa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:45.229909 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-7f3143c6-d04a-49f5-b138-38e02e5b3fa6 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:41:45.231113 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-7f3143c6-d04a-49f5-b138-38e02e5b3fa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:45.298943 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 749/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 17:41:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:41:45.476548 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df5dceb-c9aa-49fc-a93c-0cef9ec4f411 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:45.478287 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-1df5dceb-c9aa-49fc-a93c-0cef9ec4f411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:45.568553 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df5dceb-c9aa-49fc-a93c-0cef9ec4f411 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:45.574576 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cd5a91-beb9-4fce-876b-fd5699d2c64a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f86c16179a70474a817be9fda33ece9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:45.575221 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cd5a91-beb9-4fce-876b-fd5699d2c64a service nova] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:45.575502 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cd5a91-beb9-4fce-876b-fd5699d2c64a service nova] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:45.575795 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cd5a91-beb9-4fce-876b-fd5699d2c64a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:45.584262 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cd5a91-beb9-4fce-876b-fd5699d2c64a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:45.585866 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cd5a91-beb9-4fce-876b-fd5699d2c64a service nova] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:45.595145 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 764/1513] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:41:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:41:46.494307 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172b81ae-e397-4c9a-be75-a395f9d3a5a4 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:46.494900 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172b81ae-e397-4c9a-be75-a395f9d3a5a4 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:46.494900 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172b81ae-e397-4c9a-be75-a395f9d3a5a4 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:46.496297 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 750/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 17:41:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:41:46.501139 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce414dd0-db4e-49e2-a014-e2de3792829f None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:46.501576 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce414dd0-db4e-49e2-a014-e2de3792829f None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:46.501687 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce414dd0-db4e-49e2-a014-e2de3792829f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:46.521232 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-ce414dd0-db4e-49e2-a014-e2de3792829f None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:41:46.522434 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-ce414dd0-db4e-49e2-a014-e2de3792829f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:46.530306 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 765/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 17:41:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:41:46.682679 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172b81ae-e397-4c9a-be75-a395f9d3a5a4 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:46.697925 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172b81ae-e397-4c9a-be75-a395f9d3a5a4 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:46.703346 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6098694b-bb64-49f7-88e2-e33a99a5a974 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f86c16179a70474a817be9fda33ece9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:46.703980 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6098694b-bb64-49f7-88e2-e33a99a5a974 service nova] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:46.704266 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6098694b-bb64-49f7-88e2-e33a99a5a974 service nova] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:46.704581 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6098694b-bb64-49f7-88e2-e33a99a5a974 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:46.712956 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6098694b-bb64-49f7-88e2-e33a99a5a974 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:46.714555 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6098694b-bb64-49f7-88e2-e33a99a5a974 service nova] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:46.724330 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 751/1516] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:41:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:41:47.131030 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2854f6-82b5-40a6-b2c7-a897b33ebdc4 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:47.131242 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2854f6-82b5-40a6-b2c7-a897b33ebdc4 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:47.131498 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2854f6-82b5-40a6-b2c7-a897b33ebdc4 None None] PATH_INFO: `/v3` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:47.132933 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 766/1517] 10.222.0.15 () {54 vars in 925 bytes} [Thu Jun 16 17:41:47 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:41:47.140185 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e263dbe1-1092-4e79-93ba-c971650702e1 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:47.140499 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e263dbe1-1092-4e79-93ba-c971650702e1 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:47.140778 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e263dbe1-1092-4e79-93ba-c971650702e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:47.159263 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-e263dbe1-1092-4e79-93ba-c971650702e1 None None] MFA Rules not processed for user `0570d8d9508d4546ae897276f7d181d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:41:47.160518 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-e263dbe1-1092-4e79-93ba-c971650702e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:47.222259 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 752/1518] 10.222.0.15 () {58 vars in 1015 bytes} [Thu Jun 16 17:41:47 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:41:56.469872 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81a189d-ecd3-43f8-ad19-b293eeebbd2d None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:56.470393 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81a189d-ecd3-43f8-ad19-b293eeebbd2d None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:56.470393 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81a189d-ecd3-43f8-ad19-b293eeebbd2d None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:56.471712 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 767/1519] 10.222.0.42 () {54 vars in 960 bytes} [Thu Jun 16 17:41:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:41:56.477392 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f87aee2-2790-4f5f-844f-f37bd66d3f90 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:56.477782 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f87aee2-2790-4f5f-844f-f37bd66d3f90 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:56.477978 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f87aee2-2790-4f5f-844f-f37bd66d3f90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:56.496870 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-0f87aee2-2790-4f5f-844f-f37bd66d3f90 None None] MFA Rules not processed for user `0570d8d9508d4546ae897276f7d181d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:41:56.498066 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-0f87aee2-2790-4f5f-844f-f37bd66d3f90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:56.507063 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 753/1520] 10.222.0.42 () {58 vars in 1057 bytes} [Thu Jun 16 17:41:56 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:41:56.514820 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81a189d-ecd3-43f8-ad19-b293eeebbd2d None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:56.515489 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=37 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:41:56.529840 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81a189d-ecd3-43f8-ad19-b293eeebbd2d None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:56.534799 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1353652-8cf9-455a-bc72-650047594948 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0570d8d9508d4546ae897276f7d181d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:56.536013 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1353652-8cf9-455a-bc72-650047594948 service placement] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:56.536296 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1353652-8cf9-455a-bc72-650047594948 service placement] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:56.536608 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1353652-8cf9-455a-bc72-650047594948 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:56.545061 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1353652-8cf9-455a-bc72-650047594948 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:56.546633 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1353652-8cf9-455a-bc72-650047594948 service placement] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:56.556953 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 768/1521] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:41:56 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:41:58.037723 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38475f10-d5f7-4899-b5fa-c9d286b34e5c None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:58.038312 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38475f10-d5f7-4899-b5fa-c9d286b34e5c None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:58.038312 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38475f10-d5f7-4899-b5fa-c9d286b34e5c None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:58.039700 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 754/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 17:41:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:41:58.044366 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3105451a-ddc1-4950-ae21-7243fe5ec83a None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:58.044808 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3105451a-ddc1-4950-ae21-7243fe5ec83a None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:58.044947 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3105451a-ddc1-4950-ae21-7243fe5ec83a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:58.063622 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-3105451a-ddc1-4950-ae21-7243fe5ec83a None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:41:58.064863 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-3105451a-ddc1-4950-ae21-7243fe5ec83a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:58.073535 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 769/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 17:41:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:41:58.227475 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38475f10-d5f7-4899-b5fa-c9d286b34e5c None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:58.242806 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38475f10-d5f7-4899-b5fa-c9d286b34e5c None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:58.248324 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c007ac9-5238-43a4-a204-fca6a763ede7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f86c16179a70474a817be9fda33ece9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:58.248996 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c007ac9-5238-43a4-a204-fca6a763ede7 service nova] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:58.249284 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c007ac9-5238-43a4-a204-fca6a763ede7 service nova] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:58.249565 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c007ac9-5238-43a4-a204-fca6a763ede7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:58.258817 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c007ac9-5238-43a4-a204-fca6a763ede7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:58.260432 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c007ac9-5238-43a4-a204-fca6a763ede7 service nova] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:58.269912 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 755/1524] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:41:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:41:59.146217 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766b2693-7553-4484-864e-cb681bf6460d None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:59.146735 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766b2693-7553-4484-864e-cb681bf6460d None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:59.146735 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766b2693-7553-4484-864e-cb681bf6460d None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:59.148126 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 770/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 17:41:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:41:59.153173 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f89b40-28ad-495e-9097-68eef0201681 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:59.153560 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f89b40-28ad-495e-9097-68eef0201681 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:59.153738 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f89b40-28ad-495e-9097-68eef0201681 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:59.172468 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-f5f89b40-28ad-495e-9097-68eef0201681 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:41:59.173659 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-f5f89b40-28ad-495e-9097-68eef0201681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:59.181669 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 756/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 17:41:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:41:59.336352 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766b2693-7553-4484-864e-cb681bf6460d None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:59.352203 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766b2693-7553-4484-864e-cb681bf6460d None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:59.357870 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18993437-9ed0-4390-a527-f84ef5d56209 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f86c16179a70474a817be9fda33ece9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:59.358511 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18993437-9ed0-4390-a527-f84ef5d56209 service nova] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:59.358799 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18993437-9ed0-4390-a527-f84ef5d56209 service nova] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:59.359110 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18993437-9ed0-4390-a527-f84ef5d56209 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:59.368225 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18993437-9ed0-4390-a527-f84ef5d56209 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:59.369881 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18993437-9ed0-4390-a527-f84ef5d56209 service nova] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:59.379411 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 771/1527] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:41:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:42:03.332567 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69258c4f-52e0-4266-a319-b8b08d4dc3d3 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:03.333120 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69258c4f-52e0-4266-a319-b8b08d4dc3d3 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:03.333607 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69258c4f-52e0-4266-a319-b8b08d4dc3d3 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:03.335094 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 757/1528] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 17:42:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:42:03.340357 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced07480-2778-4062-92b3-2c65bf4205a1 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:03.340838 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced07480-2778-4062-92b3-2c65bf4205a1 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:03.340981 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced07480-2778-4062-92b3-2c65bf4205a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:03.358860 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-ced07480-2778-4062-92b3-2c65bf4205a1 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:42:03.360178 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-ced07480-2778-4062-92b3-2c65bf4205a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:42:03.368793 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 772/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 17:42:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:42:03.405805 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69258c4f-52e0-4266-a319-b8b08d4dc3d3 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:42:03.407479 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-69258c4f-52e0-4266-a319-b8b08d4dc3d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:42:03.463896 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69258c4f-52e0-4266-a319-b8b08d4dc3d3 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:42:03.467718 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1b3c07-9621-4538-a3a5-27cf0a906f66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:42:03.468454 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1b3c07-9621-4538-a3a5-27cf0a906f66 service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:03.468742 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1b3c07-9621-4538-a3a5-27cf0a906f66 service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:03.469019 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1b3c07-9621-4538-a3a5-27cf0a906f66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:03.479035 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1b3c07-9621-4538-a3a5-27cf0a906f66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:42:03.480339 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1b3c07-9621-4538-a3a5-27cf0a906f66 service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:42:03.485338 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:42:03.491752 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 758/1530] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:42:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:42:05.658589 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c18b0e-f6c9-48f3-80f7-f2f703308a3d None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:05.659203 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c18b0e-f6c9-48f3-80f7-f2f703308a3d None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:05.659203 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c18b0e-f6c9-48f3-80f7-f2f703308a3d None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:05.661225 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 773/1531] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 17:42:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:42:05.666421 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87772f0-247e-4381-bb91-edfd8a48dfba None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:05.666869 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87772f0-247e-4381-bb91-edfd8a48dfba None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:05.666982 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87772f0-247e-4381-bb91-edfd8a48dfba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:05.685060 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-c87772f0-247e-4381-bb91-edfd8a48dfba None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:42:05.686258 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-c87772f0-247e-4381-bb91-edfd8a48dfba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:42:05.694242 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 759/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 17:42:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:42:05.730310 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c18b0e-f6c9-48f3-80f7-f2f703308a3d None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:42:05.741543 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c18b0e-f6c9-48f3-80f7-f2f703308a3d None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:42:05.745081 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189f4064-9368-4c48-8db9-e2835ac6bb2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:42:05.745709 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189f4064-9368-4c48-8db9-e2835ac6bb2b service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:05.745993 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189f4064-9368-4c48-8db9-e2835ac6bb2b service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:05.746275 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189f4064-9368-4c48-8db9-e2835ac6bb2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:05.756725 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189f4064-9368-4c48-8db9-e2835ac6bb2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:42:05.758162 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189f4064-9368-4c48-8db9-e2835ac6bb2b service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:42:05.767701 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 774/1533] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:42:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:42:09.402572 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5944ca31-9de7-4778-b359-4e26c3b065ca None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:09.403277 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5944ca31-9de7-4778-b359-4e26c3b065ca None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:09.403277 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5944ca31-9de7-4778-b359-4e26c3b065ca None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:09.404797 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 760/1534] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 17:42:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:42:09.411758 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ffab09-2449-4457-ae8e-f83dc07f6912 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:09.412229 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ffab09-2449-4457-ae8e-f83dc07f6912 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:09.412324 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ffab09-2449-4457-ae8e-f83dc07f6912 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:09.435062 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-f0ffab09-2449-4457-ae8e-f83dc07f6912 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:42:09.436502 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-f0ffab09-2449-4457-ae8e-f83dc07f6912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:42:09.446630 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 775/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 17:42:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:42:09.722640 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5944ca31-9de7-4778-b359-4e26c3b065ca None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:42:09.724421 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-5944ca31-9de7-4778-b359-4e26c3b065ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:42:09.782267 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5944ca31-9de7-4778-b359-4e26c3b065ca None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:42:09.786654 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1eb501-b3c9-47f6-bf59-11f83d3aeb4d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24d5e2637b7043e08313909d5b06e329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:42:09.787417 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1eb501-b3c9-47f6-bf59-11f83d3aeb4d service cinder] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:09.787706 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1eb501-b3c9-47f6-bf59-11f83d3aeb4d service cinder] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:09.787987 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1eb501-b3c9-47f6-bf59-11f83d3aeb4d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:09.797102 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1eb501-b3c9-47f6-bf59-11f83d3aeb4d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:42:09.798533 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1eb501-b3c9-47f6-bf59-11f83d3aeb4d service cinder] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:42:09.808007 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 761/1536] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:42:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:42:11.966820 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf01996-2525-46ca-a47d-68d4acb54563 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:11.967377 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf01996-2525-46ca-a47d-68d4acb54563 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:11.967476 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf01996-2525-46ca-a47d-68d4acb54563 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:11.968819 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 776/1537] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 17:42:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:42:11.974420 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e552b79-a241-4d74-8d8f-7d0430ce1e70 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:11.974852 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e552b79-a241-4d74-8d8f-7d0430ce1e70 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:11.974981 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e552b79-a241-4d74-8d8f-7d0430ce1e70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:11.993314 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-1e552b79-a241-4d74-8d8f-7d0430ce1e70 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:42:11.994500 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-1e552b79-a241-4d74-8d8f-7d0430ce1e70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:42:12.002354 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 762/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 17:42:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:42:12.269066 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf01996-2525-46ca-a47d-68d4acb54563 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:42:12.271730 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-acf01996-2525-46ca-a47d-68d4acb54563 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:42:12.285127 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf01996-2525-46ca-a47d-68d4acb54563 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:42:12.289098 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b15cc3-d789-4aaf-9881-d5a8af143425 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24d5e2637b7043e08313909d5b06e329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:42:12.289742 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b15cc3-d789-4aaf-9881-d5a8af143425 service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:42:12.290021 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b15cc3-d789-4aaf-9881-d5a8af143425 service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:42:12.290501 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b15cc3-d789-4aaf-9881-d5a8af143425 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:42:12.301702 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b15cc3-d789-4aaf-9881-d5a8af143425 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:42:12.303078 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b15cc3-d789-4aaf-9881-d5a8af143425 service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:42:12.312945 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 777/1539] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:42:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:42:21.987452 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=32 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:42:22.308634 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=33 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:43:44.926967 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4084760-1b0c-4a91-a423-2f736b498531 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:44.927939 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4084760-1b0c-4a91-a423-2f736b498531 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:44.928404 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4084760-1b0c-4a91-a423-2f736b498531 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:44.959747 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-f4084760-1b0c-4a91-a423-2f736b498531 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:43:44.962308 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-f4084760-1b0c-4a91-a423-2f736b498531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:43:44.971272 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 763/1540] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:43:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:43:44.982407 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f34b2a-9f29-4a6c-aa24-0b8ae39ffb0c None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:44.983019 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f34b2a-9f29-4a6c-aa24-0b8ae39ffb0c None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:44.983239 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f34b2a-9f29-4a6c-aa24-0b8ae39ffb0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:45.029818 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-f8f34b2a-9f29-4a6c-aa24-0b8ae39ffb0c None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:43:45.032601 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-f8f34b2a-9f29-4a6c-aa24-0b8ae39ffb0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:43:45.046362 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 778/1541] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:43:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:43:45.054691 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4084760-1b0c-4a91-a423-2f736b498531 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:45.070097 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4084760-1b0c-4a91-a423-2f736b498531 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:45.083020 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1cfbb7-6db7-4be5-8526-661245b32b29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:45.084145 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1cfbb7-6db7-4be5-8526-661245b32b29 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:45.084721 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1cfbb7-6db7-4be5-8526-661245b32b29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:45.085236 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1cfbb7-6db7-4be5-8526-661245b32b29 admin admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:45.086230 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1cfbb7-6db7-4be5-8526-661245b32b29 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:43:45.086783 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1cfbb7-6db7-4be5-8526-661245b32b29 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:45.088799 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1cfbb7-6db7-4be5-8526-661245b32b29 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:45.093098 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b875909-d5e2-42d1-9296-bc8e88afc446 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:45.094388 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b875909-d5e2-42d1-9296-bc8e88afc446 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:45.094981 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b875909-d5e2-42d1-9296-bc8e88afc446 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:45.114894 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 764/1542] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jun 16 17:43:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:43:45.123096 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1cfbb7-6db7-4be5-8526-661245b32b29 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:45.129378 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-0b875909-d5e2-42d1-9296-bc8e88afc446 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:43:45.130875 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-0b875909-d5e2-42d1-9296-bc8e88afc446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:43:45.139199 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1cfbb7-6db7-4be5-8526-661245b32b29 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:45.149522 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 779/1543] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:43:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:43:45.150125 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a943fa-a26e-4cd0-91ef-b3b3dfc4fef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:45.150812 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a943fa-a26e-4cd0-91ef-b3b3dfc4fef1 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:45.151099 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a943fa-a26e-4cd0-91ef-b3b3dfc4fef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:45.151378 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a943fa-a26e-4cd0-91ef-b3b3dfc4fef1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:45.152137 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a943fa-a26e-4cd0-91ef-b3b3dfc4fef1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:45.153257 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: WARNING py.warnings [None req-89a943fa-a26e-4cd0-91ef-b3b3dfc4fef1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 17:43:45.153257 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: warnings.warn(msg) Jun 16 17:43:45.153257 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: Jun 16 17:43:45.154447 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a943fa-a26e-4cd0-91ef-b3b3dfc4fef1 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:45.161231 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84272e68-d09b-4dbf-bb87-3b33808fdb02 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:45.161833 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84272e68-d09b-4dbf-bb87-3b33808fdb02 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:45.162344 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84272e68-d09b-4dbf-bb87-3b33808fdb02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:45.201476 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-84272e68-d09b-4dbf-bb87-3b33808fdb02 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:43:45.202733 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-84272e68-d09b-4dbf-bb87-3b33808fdb02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:43:45.216086 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 780/1544] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:43:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:43:45.217657 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb94288-4ebb-4b38-9d25-0ac99362d4ff None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:45.217952 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb94288-4ebb-4b38-9d25-0ac99362d4ff None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:45.218231 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb94288-4ebb-4b38-9d25-0ac99362d4ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:45.228535 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 765/1545] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jun 16 17:43:45 2022] POST /identity/v3/projects => generated 377 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:43:45.229948 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb87b6d-9e19-4f2b-a655-f39cda6f015c None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:45.230248 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb87b6d-9e19-4f2b-a655-f39cda6f015c None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:45.230829 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb87b6d-9e19-4f2b-a655-f39cda6f015c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:45.248837 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-ffb94288-4ebb-4b38-9d25-0ac99362d4ff None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:43:45.251280 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-ffb94288-4ebb-4b38-9d25-0ac99362d4ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:43:45.252150 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-eeb87b6d-9e19-4f2b-a655-f39cda6f015c None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:43:45.253554 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-eeb87b6d-9e19-4f2b-a655-f39cda6f015c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:43:45.325395 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 781/1546] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:43:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:43:45.327296 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e33452-788d-47d0-97f6-bcd45b7353ba None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:45.329293 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e33452-788d-47d0-97f6-bcd45b7353ba None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:45.330119 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e33452-788d-47d0-97f6-bcd45b7353ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:45.330269 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 766/1547] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:43:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:43:45.334469 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4707d3-8a50-405a-ada1-a05d3e842b09 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:45.335019 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4707d3-8a50-405a-ada1-a05d3e842b09 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:45.335650 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4707d3-8a50-405a-ada1-a05d3e842b09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:45.350395 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-e1e33452-788d-47d0-97f6-bcd45b7353ba None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:43:45.351620 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-e1e33452-788d-47d0-97f6-bcd45b7353ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:43:45.357863 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-4d4707d3-8a50-405a-ada1-a05d3e842b09 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:43:45.359033 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-4d4707d3-8a50-405a-ada1-a05d3e842b09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:43:45.360307 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 782/1548] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:43:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:43:45.361412 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861adaec-5695-42ae-b1e6-46d80b3d3cac None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:45.361699 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861adaec-5695-42ae-b1e6-46d80b3d3cac None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:45.361992 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861adaec-5695-42ae-b1e6-46d80b3d3cac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:45.377976 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 767/1549] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:43:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:43:45.378638 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4707d3-8a50-405a-ada1-a05d3e842b09 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:45.387166 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-861adaec-5695-42ae-b1e6-46d80b3d3cac None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:43:45.388513 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-861adaec-5695-42ae-b1e6-46d80b3d3cac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:43:45.389933 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4707d3-8a50-405a-ada1-a05d3e842b09 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:45.397513 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986b0549-6e42-423b-bb77-a7613d5281e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:45.398183 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986b0549-6e42-423b-bb77-a7613d5281e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:45.398489 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986b0549-6e42-423b-bb77-a7613d5281e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:45.398735 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986b0549-6e42-423b-bb77-a7613d5281e3 admin admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:45.399420 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986b0549-6e42-423b-bb77-a7613d5281e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:45.400121 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 783/1550] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:43:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:43:45.403594 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e818e9-497b-41c7-a7a6-2d1b7a271dfa None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:45.403736 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986b0549-6e42-423b-bb77-a7613d5281e3 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:45.403835 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e818e9-497b-41c7-a7a6-2d1b7a271dfa None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:45.403835 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e818e9-497b-41c7-a7a6-2d1b7a271dfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:45.431919 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-19e818e9-497b-41c7-a7a6-2d1b7a271dfa None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:43:45.433384 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-19e818e9-497b-41c7-a7a6-2d1b7a271dfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:43:45.442113 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 768/1551] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jun 16 17:43:45 2022] POST /identity/v3/users => generated 289 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:43:45.443204 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986b0549-6e42-423b-bb77-a7613d5281e3 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:45.447546 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 784/1552] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:43:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:43:45.448808 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b5f34d-cc27-46e3-a148-ac8915b98cb0 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:45.449101 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b5f34d-cc27-46e3-a148-ac8915b98cb0 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:45.449379 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b5f34d-cc27-46e3-a148-ac8915b98cb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:45.455970 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986b0549-6e42-423b-bb77-a7613d5281e3 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:45.461261 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ffffd4-0e39-4278-b3bf-c654f99832df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:45.462079 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ffffd4-0e39-4278-b3bf-c654f99832df admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:45.462233 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ffffd4-0e39-4278-b3bf-c654f99832df admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:45.462463 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ffffd4-0e39-4278-b3bf-c654f99832df admin admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:45.463194 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ffffd4-0e39-4278-b3bf-c654f99832df admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:43:45.463551 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ffffd4-0e39-4278-b3bf-c654f99832df admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:45.465260 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ffffd4-0e39-4278-b3bf-c654f99832df admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:45.470235 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-b0b5f34d-cc27-46e3-a148-ac8915b98cb0 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:43:45.471456 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-b0b5f34d-cc27-46e3-a148-ac8915b98cb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:43:45.477770 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 769/1553] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jun 16 17:43:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:43:45.479186 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea32315f-870e-46c9-a874-a25550d26f0c None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:45.479669 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 785/1554] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:43:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:43:45.480852 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22ab2aa-3802-445c-8aae-63ba21b78a64 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:45.481138 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22ab2aa-3802-445c-8aae-63ba21b78a64 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:45.481418 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22ab2aa-3802-445c-8aae-63ba21b78a64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:45.483188 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea32315f-870e-46c9-a874-a25550d26f0c None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:45.483525 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea32315f-870e-46c9-a874-a25550d26f0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:45.499471 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-c22ab2aa-3802-445c-8aae-63ba21b78a64 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:43:45.501590 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-ea32315f-870e-46c9-a874-a25550d26f0c None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:43:45.501843 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-c22ab2aa-3802-445c-8aae-63ba21b78a64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:43:45.502746 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-ea32315f-870e-46c9-a874-a25550d26f0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:43:45.510001 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 786/1555] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:43:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:43:45.511225 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e60648c-c413-4cef-81b0-f6fe4208e0d9 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:45.511514 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e60648c-c413-4cef-81b0-f6fe4208e0d9 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:45.511828 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 770/1556] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:43:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:43:45.512714 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea32315f-870e-46c9-a874-a25550d26f0c None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:45.517028 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e60648c-c413-4cef-81b0-f6fe4208e0d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:45.525749 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea32315f-870e-46c9-a874-a25550d26f0c None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:45.531808 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9a57e6-c02e-4772-a4b1-1752b5391938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:45.532507 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9a57e6-c02e-4772-a4b1-1752b5391938 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:45.533034 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9a57e6-c02e-4772-a4b1-1752b5391938 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:45.533313 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9a57e6-c02e-4772-a4b1-1752b5391938 admin admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:45.534043 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9a57e6-c02e-4772-a4b1-1752b5391938 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:43:45.534348 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9a57e6-c02e-4772-a4b1-1752b5391938 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:45.535599 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-4e60648c-c413-4cef-81b0-f6fe4208e0d9 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:43:45.535947 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9a57e6-c02e-4772-a4b1-1752b5391938 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:45.536898 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-4e60648c-c413-4cef-81b0-f6fe4208e0d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:43:45.545403 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 787/1557] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:43:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:43:45.546242 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e60648c-c413-4cef-81b0-f6fe4208e0d9 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:45.549804 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 771/1558] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jun 16 17:43:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:43:45.551288 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9739092-d606-4b7c-af01-595e4e089fe2 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:45.551583 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9739092-d606-4b7c-af01-595e4e089fe2 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:45.551878 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9739092-d606-4b7c-af01-595e4e089fe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:45.556494 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e60648c-c413-4cef-81b0-f6fe4208e0d9 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:45.561037 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b41857-fc82-4913-8bca-eae9f2dd593f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:45.561697 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b41857-fc82-4913-8bca-eae9f2dd593f admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:45.561981 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b41857-fc82-4913-8bca-eae9f2dd593f admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:45.562259 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b41857-fc82-4913-8bca-eae9f2dd593f admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:45.564287 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b41857-fc82-4913-8bca-eae9f2dd593f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:45.565906 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b41857-fc82-4913-8bca-eae9f2dd593f admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:45.570361 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-b9739092-d606-4b7c-af01-595e4e089fe2 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:43:45.571569 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-b9739092-d606-4b7c-af01-595e4e089fe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:43:45.576065 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 788/1559] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:43:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:43:45.576877 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b41857-fc82-4913-8bca-eae9f2dd593f admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:45.580743 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 772/1560] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:43:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:43:45.584170 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9739092-d606-4b7c-af01-595e4e089fe2 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:45.588736 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b41857-fc82-4913-8bca-eae9f2dd593f admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:45.593305 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4273ce82-304f-4452-a8d1-111f4f63fad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:45.593723 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9739092-d606-4b7c-af01-595e4e089fe2 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:45.594563 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4273ce82-304f-4452-a8d1-111f4f63fad2 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:45.594851 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4273ce82-304f-4452-a8d1-111f4f63fad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:45.595133 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4273ce82-304f-4452-a8d1-111f4f63fad2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:45.595850 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4273ce82-304f-4452-a8d1-111f4f63fad2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:43:45.596158 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4273ce82-304f-4452-a8d1-111f4f63fad2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:45.597204 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: WARNING py.warnings [None req-4273ce82-304f-4452-a8d1-111f4f63fad2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 16 17:43:45.597204 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: warnings.warn(msg) Jun 16 17:43:45.597204 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: Jun 16 17:43:45.598083 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4273ce82-304f-4452-a8d1-111f4f63fad2 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:45.598303 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c584cd9b-97c1-4cba-8863-839c3bb5f432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:45.598949 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c584cd9b-97c1-4cba-8863-839c3bb5f432 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:45.599233 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c584cd9b-97c1-4cba-8863-839c3bb5f432 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:45.599568 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c584cd9b-97c1-4cba-8863-839c3bb5f432 admin admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:45.600698 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c584cd9b-97c1-4cba-8863-839c3bb5f432 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:45.602173 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c584cd9b-97c1-4cba-8863-839c3bb5f432 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:45.610540 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 789/1561] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jun 16 17:43:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:43:45.611790 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cfe5af-ceb3-4d93-9467-6a4684268472 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:45.612062 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cfe5af-ceb3-4d93-9467-6a4684268472 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:45.612332 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cfe5af-ceb3-4d93-9467-6a4684268472 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:45.630436 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 773/1562] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jun 16 17:43:45 2022] POST /identity/v3/projects => generated 361 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:43:45.631356 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c584cd9b-97c1-4cba-8863-839c3bb5f432 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:45.631732 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-c3cfe5af-ceb3-4d93-9467-6a4684268472 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:43:45.633266 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-c3cfe5af-ceb3-4d93-9467-6a4684268472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:43:45.669998 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:43:45.696383 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 790/1563] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:43:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:43:45.697288 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cfe5af-ceb3-4d93-9467-6a4684268472 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:45.699079 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c584cd9b-97c1-4cba-8863-839c3bb5f432 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:45.704177 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9497ee9a-66b2-4ba5-876a-7fc31d732d36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:45.704865 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9497ee9a-66b2-4ba5-876a-7fc31d732d36 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:45.705149 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9497ee9a-66b2-4ba5-876a-7fc31d732d36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:45.705426 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9497ee9a-66b2-4ba5-876a-7fc31d732d36 admin admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:45.706143 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9497ee9a-66b2-4ba5-876a-7fc31d732d36 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:43:45.706446 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9497ee9a-66b2-4ba5-876a-7fc31d732d36 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:45.707539 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cfe5af-ceb3-4d93-9467-6a4684268472 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:45.708035 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9497ee9a-66b2-4ba5-876a-7fc31d732d36 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:45.712171 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c073e73-013f-4a81-8c6b-2dccd096d8a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:45.712879 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c073e73-013f-4a81-8c6b-2dccd096d8a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:45.713163 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c073e73-013f-4a81-8c6b-2dccd096d8a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:45.713439 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c073e73-013f-4a81-8c6b-2dccd096d8a3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:45.714151 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c073e73-013f-4a81-8c6b-2dccd096d8a3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:43:45.714455 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c073e73-013f-4a81-8c6b-2dccd096d8a3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:45.716050 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c073e73-013f-4a81-8c6b-2dccd096d8a3 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:45.721108 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 774/1564] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jun 16 17:43:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:43:45.721927 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9497ee9a-66b2-4ba5-876a-7fc31d732d36 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:45.734817 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 791/1565] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jun 16 17:43:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:43:45.734817 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c073e73-013f-4a81-8c6b-2dccd096d8a3 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:45.735085 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9497ee9a-66b2-4ba5-876a-7fc31d732d36 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:45.738820 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5ff99b-a9a9-421e-a799-18f90b24d9e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:45.739596 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5ff99b-a9a9-421e-a799-18f90b24d9e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:45.739880 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5ff99b-a9a9-421e-a799-18f90b24d9e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:45.740302 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5ff99b-a9a9-421e-a799-18f90b24d9e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:45.740927 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:43:45.741062 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5ff99b-a9a9-421e-a799-18f90b24d9e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:45.742936 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5ff99b-a9a9-421e-a799-18f90b24d9e1 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:45.745025 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c073e73-013f-4a81-8c6b-2dccd096d8a3 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:45.750704 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ca6307-2188-4e32-b05d-3b2f5e390e22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:45.751473 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ca6307-2188-4e32-b05d-3b2f5e390e22 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:45.751752 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ca6307-2188-4e32-b05d-3b2f5e390e22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:45.752029 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ca6307-2188-4e32-b05d-3b2f5e390e22 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:45.752794 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ca6307-2188-4e32-b05d-3b2f5e390e22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:45.754394 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ca6307-2188-4e32-b05d-3b2f5e390e22 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:45.770201 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 792/1566] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:43:45 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:43:45.770201 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ca6307-2188-4e32-b05d-3b2f5e390e22 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:45.775520 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 775/1567] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jun 16 17:43:45 2022] POST /identity/v3/projects => generated 375 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:43:45.776468 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5ff99b-a9a9-421e-a799-18f90b24d9e1 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:45.782427 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ca6307-2188-4e32-b05d-3b2f5e390e22 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:45.846414 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5ff99b-a9a9-421e-a799-18f90b24d9e1 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:45.851010 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f573273-851d-42e4-9ca1-faf0d5791bc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:45.851491 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c6f402-d3f6-4f37-b76e-16dd4ad81a20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:45.851673 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f573273-851d-42e4-9ca1-faf0d5791bc9 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:45.851953 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f573273-851d-42e4-9ca1-faf0d5791bc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:45.852153 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c6f402-d3f6-4f37-b76e-16dd4ad81a20 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:45.852265 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f573273-851d-42e4-9ca1-faf0d5791bc9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:45.852466 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c6f402-d3f6-4f37-b76e-16dd4ad81a20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:45.852765 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c6f402-d3f6-4f37-b76e-16dd4ad81a20 admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:45.852943 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f573273-851d-42e4-9ca1-faf0d5791bc9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:45.853607 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c6f402-d3f6-4f37-b76e-16dd4ad81a20 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:43:45.853911 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c6f402-d3f6-4f37-b76e-16dd4ad81a20 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:45.854824 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f573273-851d-42e4-9ca1-faf0d5791bc9 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:45.855475 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c6f402-d3f6-4f37-b76e-16dd4ad81a20 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:45.869437 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 793/1568] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jun 16 17:43:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:43:45.870246 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c6f402-d3f6-4f37-b76e-16dd4ad81a20 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:45.882147 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 776/1569] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jun 16 17:43:45 2022] POST /identity/v3/projects => generated 385 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:43:45.883493 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f573273-851d-42e4-9ca1-faf0d5791bc9 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:45.888983 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c6f402-d3f6-4f37-b76e-16dd4ad81a20 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:45.947114 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f573273-851d-42e4-9ca1-faf0d5791bc9 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:45.948675 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6a19a3-7e9b-429d-b070-35b49bb39501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:45.949352 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6a19a3-7e9b-429d-b070-35b49bb39501 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:45.949704 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6a19a3-7e9b-429d-b070-35b49bb39501 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:45.949935 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6a19a3-7e9b-429d-b070-35b49bb39501 admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:45.950750 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6a19a3-7e9b-429d-b070-35b49bb39501 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:45.951898 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: WARNING py.warnings [None req-7a6a19a3-7e9b-429d-b070-35b49bb39501 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 17:43:45.951898 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: warnings.warn(msg) Jun 16 17:43:45.951898 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: Jun 16 17:43:45.952362 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3be1204-cf4b-46ce-a774-363dd8fa1cd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:45.953129 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3be1204-cf4b-46ce-a774-363dd8fa1cd8 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:45.953344 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6a19a3-7e9b-429d-b070-35b49bb39501 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:45.953554 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3be1204-cf4b-46ce-a774-363dd8fa1cd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:45.953898 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3be1204-cf4b-46ce-a774-363dd8fa1cd8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:45.954573 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3be1204-cf4b-46ce-a774-363dd8fa1cd8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:43:45.954885 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3be1204-cf4b-46ce-a774-363dd8fa1cd8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:45.956512 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3be1204-cf4b-46ce-a774-363dd8fa1cd8 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:45.969371 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 777/1570] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jun 16 17:43:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:43:45.970193 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3be1204-cf4b-46ce-a774-363dd8fa1cd8 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:45.978108 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 794/1571] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jun 16 17:43:45 2022] POST /identity/v3/users => generated 281 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:43:45.979001 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6a19a3-7e9b-429d-b070-35b49bb39501 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:45.987599 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3be1204-cf4b-46ce-a774-363dd8fa1cd8 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:45.989325 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6a19a3-7e9b-429d-b070-35b49bb39501 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:45.992516 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5bf60e-6f2e-41ce-9a0e-fb6de84f0be5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:45.993176 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5bf60e-6f2e-41ce-9a0e-fb6de84f0be5 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:45.993477 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5bf60e-6f2e-41ce-9a0e-fb6de84f0be5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:45.993751 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5bf60e-6f2e-41ce-9a0e-fb6de84f0be5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:45.993853 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b02ca0d-2773-41fb-beed-2216fc2597a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:45.994458 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5bf60e-6f2e-41ce-9a0e-fb6de84f0be5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:45.994570 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b02ca0d-2773-41fb-beed-2216fc2597a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:45.994755 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b02ca0d-2773-41fb-beed-2216fc2597a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:45.995022 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b02ca0d-2773-41fb-beed-2216fc2597a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:45.995710 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b02ca0d-2773-41fb-beed-2216fc2597a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:45.996345 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5bf60e-6f2e-41ce-9a0e-fb6de84f0be5 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:45.997609 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b02ca0d-2773-41fb-beed-2216fc2597a5 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:46.024407 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 778/1572] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jun 16 17:43:45 2022] POST /identity/v3/projects => generated 393 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:43:46.025141 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5bf60e-6f2e-41ce-9a0e-fb6de84f0be5 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:46.028670 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 795/1573] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jun 16 17:43:45 2022] POST /identity/v3/projects => generated 367 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:43:46.028670 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b02ca0d-2773-41fb-beed-2216fc2597a5 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:46.031720 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:43:46.086587 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:43:46.092204 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b02ca0d-2773-41fb-beed-2216fc2597a5 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:46.092576 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5bf60e-6f2e-41ce-9a0e-fb6de84f0be5 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:46.098518 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b27418-9db1-4d48-aac1-20a43fd735c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:46.098690 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d5375b-e44d-40bc-9b96-6b6126cb8e19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:46.099188 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b27418-9db1-4d48-aac1-20a43fd735c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:46.099433 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d5375b-e44d-40bc-9b96-6b6126cb8e19 admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:46.099542 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b27418-9db1-4d48-aac1-20a43fd735c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:46.099721 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d5375b-e44d-40bc-9b96-6b6126cb8e19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:46.099818 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b27418-9db1-4d48-aac1-20a43fd735c9 admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:46.099998 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d5375b-e44d-40bc-9b96-6b6126cb8e19 admin admin] PATH_INFO: `/v3/projects/dd279f49cbd84e9f926aa6372a4728d4/users/813dd0cf7e1d4724b903b358aaac1ea7/roles/0be2075946f54c8485f1c275f746478a` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:46.100423 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b27418-9db1-4d48-aac1-20a43fd735c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:46.102356 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b27418-9db1-4d48-aac1-20a43fd735c9 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:46.116993 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d5375b-e44d-40bc-9b96-6b6126cb8e19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd279f49cbd84e9f926aa6372a4728d4, user_id=813dd0cf7e1d4724b903b358aaac1ea7, role_id=0be2075946f54c8485f1c275f746478a)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:46.118149 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: WARNING py.warnings [None req-81d5375b-e44d-40bc-9b96-6b6126cb8e19 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 17:43:46.118149 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: warnings.warn(msg) Jun 16 17:43:46.118149 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: Jun 16 17:43:46.120076 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d5375b-e44d-40bc-9b96-6b6126cb8e19 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:46.123619 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 796/1574] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jun 16 17:43:46 2022] POST /identity/v3/users => generated 288 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:43:46.124606 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b27418-9db1-4d48-aac1-20a43fd735c9 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:46.135836 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b27418-9db1-4d48-aac1-20a43fd735c9 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:46.138270 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 779/1575] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:43:46 2022] PUT /identity/v3/projects/dd279f49cbd84e9f926aa6372a4728d4/users/813dd0cf7e1d4724b903b358aaac1ea7/roles/0be2075946f54c8485f1c275f746478a => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:43:46.139141 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d5375b-e44d-40bc-9b96-6b6126cb8e19 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:46.201212 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2697851d-a8ad-498b-84e7-62e7cfc573ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:46.201893 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2697851d-a8ad-498b-84e7-62e7cfc573ec admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:46.202249 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2697851d-a8ad-498b-84e7-62e7cfc573ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:46.202597 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2697851d-a8ad-498b-84e7-62e7cfc573ec admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:46.203397 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2697851d-a8ad-498b-84e7-62e7cfc573ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:46.205416 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2697851d-a8ad-498b-84e7-62e7cfc573ec admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:46.209691 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d5375b-e44d-40bc-9b96-6b6126cb8e19 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:46.214393 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bba76da-4b60-4d04-b26e-03bbe2fc8440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:46.215097 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bba76da-4b60-4d04-b26e-03bbe2fc8440 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:46.215387 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bba76da-4b60-4d04-b26e-03bbe2fc8440 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:46.215673 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bba76da-4b60-4d04-b26e-03bbe2fc8440 admin admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:46.216410 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bba76da-4b60-4d04-b26e-03bbe2fc8440 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:46.218342 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bba76da-4b60-4d04-b26e-03bbe2fc8440 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:46.233660 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 797/1576] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jun 16 17:43:46 2022] POST /identity/v3/projects => generated 387 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:43:46.235258 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2697851d-a8ad-498b-84e7-62e7cfc573ec admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:46.244745 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 780/1577] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jun 16 17:43:46 2022] POST /identity/v3/users => generated 293 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:43:46.244745 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bba76da-4b60-4d04-b26e-03bbe2fc8440 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:46.254522 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bba76da-4b60-4d04-b26e-03bbe2fc8440 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:46.296302 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2697851d-a8ad-498b-84e7-62e7cfc573ec admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:46.301628 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97acf56-77ca-46d2-86ec-b4c6ae0983bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:46.302285 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97acf56-77ca-46d2-86ec-b4c6ae0983bc admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:46.302563 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97acf56-77ca-46d2-86ec-b4c6ae0983bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:46.302836 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97acf56-77ca-46d2-86ec-b4c6ae0983bc admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:46.303520 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97acf56-77ca-46d2-86ec-b4c6ae0983bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:46.305463 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97acf56-77ca-46d2-86ec-b4c6ae0983bc admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:46.312866 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266a585d-b04a-4f51-a29b-6a6d6753d441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:46.313541 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266a585d-b04a-4f51-a29b-6a6d6753d441 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:46.313826 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266a585d-b04a-4f51-a29b-6a6d6753d441 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:46.314105 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266a585d-b04a-4f51-a29b-6a6d6753d441 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:46.314862 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266a585d-b04a-4f51-a29b-6a6d6753d441 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:46.316665 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266a585d-b04a-4f51-a29b-6a6d6753d441 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:46.327968 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 781/1578] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:43:46 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:43:46.329042 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266a585d-b04a-4f51-a29b-6a6d6753d441 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:46.335378 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 798/1579] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jun 16 17:43:46 2022] POST /identity/v3/projects => generated 405 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:43:46.335378 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97acf56-77ca-46d2-86ec-b4c6ae0983bc admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:46.338750 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266a585d-b04a-4f51-a29b-6a6d6753d441 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:46.347502 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:43:46.394673 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:43:46.400068 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305554b6-a696-47ab-b6d5-e09a894782e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:46.400782 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305554b6-a696-47ab-b6d5-e09a894782e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:46.401063 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305554b6-a696-47ab-b6d5-e09a894782e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:46.401216 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97acf56-77ca-46d2-86ec-b4c6ae0983bc admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:46.401322 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305554b6-a696-47ab-b6d5-e09a894782e6 admin admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:46.402002 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305554b6-a696-47ab-b6d5-e09a894782e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:46.403912 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305554b6-a696-47ab-b6d5-e09a894782e6 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:46.406997 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81a7689-361f-4054-89b6-cfe6dbcaa0ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:46.407744 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81a7689-361f-4054-89b6-cfe6dbcaa0ba admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:46.408026 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81a7689-361f-4054-89b6-cfe6dbcaa0ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:46.408302 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81a7689-361f-4054-89b6-cfe6dbcaa0ba admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:46.409015 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81a7689-361f-4054-89b6-cfe6dbcaa0ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:46.411064 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81a7689-361f-4054-89b6-cfe6dbcaa0ba admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:46.428405 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 782/1580] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jun 16 17:43:46 2022] POST /identity/v3/users => generated 297 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:43:46.428405 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305554b6-a696-47ab-b6d5-e09a894782e6 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:46.439233 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 799/1581] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jun 16 17:43:46 2022] POST /identity/v3/users => generated 284 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:43:46.439410 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305554b6-a696-47ab-b6d5-e09a894782e6 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:46.439533 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff5effd-d3de-405f-ad9b-ccd94dd89c17 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:46.439533 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff5effd-d3de-405f-ad9b-ccd94dd89c17 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:46.439533 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff5effd-d3de-405f-ad9b-ccd94dd89c17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:46.444438 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8b2d58-03dc-4fd5-8534-ed1f114a9959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:46.445103 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8b2d58-03dc-4fd5-8534-ed1f114a9959 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:46.445397 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8b2d58-03dc-4fd5-8534-ed1f114a9959 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:46.445669 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8b2d58-03dc-4fd5-8534-ed1f114a9959 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:46.446400 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8b2d58-03dc-4fd5-8534-ed1f114a9959 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:46.448034 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8b2d58-03dc-4fd5-8534-ed1f114a9959 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:46.455222 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-5ff5effd-d3de-405f-ad9b-ccd94dd89c17 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:43:46.456399 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-5ff5effd-d3de-405f-ad9b-ccd94dd89c17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:43:46.459411 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 783/1582] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:43:46 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:43:46.460227 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8b2d58-03dc-4fd5-8534-ed1f114a9959 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:46.467143 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 800/1583] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:43:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:43:46.467948 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff5effd-d3de-405f-ad9b-ccd94dd89c17 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:46.474482 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8b2d58-03dc-4fd5-8534-ed1f114a9959 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:46.477776 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff5effd-d3de-405f-ad9b-ccd94dd89c17 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:46.479232 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71777f4f-f557-46ba-9198-30c98d7fed80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:46.480000 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71777f4f-f557-46ba-9198-30c98d7fed80 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:46.480285 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71777f4f-f557-46ba-9198-30c98d7fed80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:46.480596 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71777f4f-f557-46ba-9198-30c98d7fed80 admin admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:46.481273 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71777f4f-f557-46ba-9198-30c98d7fed80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:46.482613 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7f4f0d-8bf1-4137-b75a-7dfb401690c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:46.483342 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7f4f0d-8bf1-4137-b75a-7dfb401690c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:46.483461 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71777f4f-f557-46ba-9198-30c98d7fed80 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:46.483754 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7f4f0d-8bf1-4137-b75a-7dfb401690c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:46.484098 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7f4f0d-8bf1-4137-b75a-7dfb401690c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:46.485526 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7f4f0d-8bf1-4137-b75a-7dfb401690c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:46.487116 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7f4f0d-8bf1-4137-b75a-7dfb401690c6 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:46.497766 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 801/1584] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:43:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:43:46.498698 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7f4f0d-8bf1-4137-b75a-7dfb401690c6 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:46.508916 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7f4f0d-8bf1-4137-b75a-7dfb401690c6 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:46.509264 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 784/1585] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jun 16 17:43:46 2022] POST /identity/v3/users => generated 294 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:43:46.510051 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71777f4f-f557-46ba-9198-30c98d7fed80 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:46.516543 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602f01bc-9b85-4863-8222-6b0cdb807ad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:46.517336 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602f01bc-9b85-4863-8222-6b0cdb807ad9 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:46.517488 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602f01bc-9b85-4863-8222-6b0cdb807ad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:46.517761 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602f01bc-9b85-4863-8222-6b0cdb807ad9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:46.518474 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602f01bc-9b85-4863-8222-6b0cdb807ad9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:46.520099 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602f01bc-9b85-4863-8222-6b0cdb807ad9 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:46.520529 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71777f4f-f557-46ba-9198-30c98d7fed80 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:46.525114 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc14265-113a-4abe-bf45-f9c7be650e22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:46.525821 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc14265-113a-4abe-bf45-f9c7be650e22 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:46.526104 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc14265-113a-4abe-bf45-f9c7be650e22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:46.526373 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc14265-113a-4abe-bf45-f9c7be650e22 admin admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:46.527054 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc14265-113a-4abe-bf45-f9c7be650e22 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:46.529083 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc14265-113a-4abe-bf45-f9c7be650e22 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:46.530191 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 802/1586] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:43:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:43:46.530951 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602f01bc-9b85-4863-8222-6b0cdb807ad9 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:46.540421 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602f01bc-9b85-4863-8222-6b0cdb807ad9 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:46.545301 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7c9819-086d-4e87-98b6-742e619bbfc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:46.545968 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7c9819-086d-4e87-98b6-742e619bbfc2 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:46.546249 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7c9819-086d-4e87-98b6-742e619bbfc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:46.546622 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7c9819-086d-4e87-98b6-742e619bbfc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:46.547801 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7c9819-086d-4e87-98b6-742e619bbfc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:46.549585 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7c9819-086d-4e87-98b6-742e619bbfc2 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:46.557289 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 785/1587] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jun 16 17:43:46 2022] POST /identity/v3/users => generated 303 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:43:46.557289 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc14265-113a-4abe-bf45-f9c7be650e22 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:46.560452 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 803/1588] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:43:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:43:46.561496 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7c9819-086d-4e87-98b6-742e619bbfc2 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:46.565445 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc14265-113a-4abe-bf45-f9c7be650e22 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:46.570011 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35979ccc-b872-4e46-871f-1a36be81cab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:46.570672 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35979ccc-b872-4e46-871f-1a36be81cab9 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:46.570961 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35979ccc-b872-4e46-871f-1a36be81cab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:46.571238 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35979ccc-b872-4e46-871f-1a36be81cab9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:46.571997 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35979ccc-b872-4e46-871f-1a36be81cab9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:46.573045 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7c9819-086d-4e87-98b6-742e619bbfc2 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:46.574026 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35979ccc-b872-4e46-871f-1a36be81cab9 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:46.577758 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7988895d-419c-4672-85cd-dc0b86f5b708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:46.578963 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7988895d-419c-4672-85cd-dc0b86f5b708 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:46.579322 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7988895d-419c-4672-85cd-dc0b86f5b708 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:46.579603 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7988895d-419c-4672-85cd-dc0b86f5b708 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:46.580317 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7988895d-419c-4672-85cd-dc0b86f5b708 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:46.581921 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7988895d-419c-4672-85cd-dc0b86f5b708 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:46.584213 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 786/1589] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:43:46 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:43:46.585054 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35979ccc-b872-4e46-871f-1a36be81cab9 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:46.586658 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-35979ccc-b872-4e46-871f-1a36be81cab9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:43:46.595272 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 804/1590] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:43:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:43:46.595272 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7988895d-419c-4672-85cd-dc0b86f5b708 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:46.604203 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7988895d-419c-4672-85cd-dc0b86f5b708 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:46.608850 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d8d06e-e0e2-44ef-8618-e278874a5353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:46.610601 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d8d06e-e0e2-44ef-8618-e278874a5353 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:46.610888 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d8d06e-e0e2-44ef-8618-e278874a5353 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:46.611186 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d8d06e-e0e2-44ef-8618-e278874a5353 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:46.611902 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d8d06e-e0e2-44ef-8618-e278874a5353 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:46.613502 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d8d06e-e0e2-44ef-8618-e278874a5353 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:46.623644 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 805/1591] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:43:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:43:46.624532 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d8d06e-e0e2-44ef-8618-e278874a5353 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:46.634920 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d8d06e-e0e2-44ef-8618-e278874a5353 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:46.639711 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd5607f-ee0f-429f-a9b0-e39133921abd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:46.640356 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd5607f-ee0f-429f-a9b0-e39133921abd admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:46.641703 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd5607f-ee0f-429f-a9b0-e39133921abd admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:46.641995 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd5607f-ee0f-429f-a9b0-e39133921abd admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:46.642706 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd5607f-ee0f-429f-a9b0-e39133921abd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:46.644278 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd5607f-ee0f-429f-a9b0-e39133921abd admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:46.644920 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35979ccc-b872-4e46-871f-1a36be81cab9 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:46.648584 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c464260d-aae5-43a5-ba57-3c0cfdb74e49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:46.649234 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c464260d-aae5-43a5-ba57-3c0cfdb74e49 service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:46.649519 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c464260d-aae5-43a5-ba57-3c0cfdb74e49 service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:46.649796 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c464260d-aae5-43a5-ba57-3c0cfdb74e49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:46.655761 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 806/1592] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:43:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:43:46.657355 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd5607f-ee0f-429f-a9b0-e39133921abd admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:46.660070 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c464260d-aae5-43a5-ba57-3c0cfdb74e49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:46.661537 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c464260d-aae5-43a5-ba57-3c0cfdb74e49 service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:46.666973 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd5607f-ee0f-429f-a9b0-e39133921abd admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:46.671474 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 787/1593] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:43:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:43:46.671593 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c408dd-210a-48ab-b006-c0e5086f982f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:46.672882 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c464260d-aae5-43a5-ba57-3c0cfdb74e49 service neutron] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:46.673606 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c408dd-210a-48ab-b006-c0e5086f982f admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:46.673896 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c408dd-210a-48ab-b006-c0e5086f982f admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:46.674169 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c408dd-210a-48ab-b006-c0e5086f982f admin admin] PATH_INFO: `/v3/projects/e093a761d7014c8d81a9a8e3202637a3/users/1226c2eb02fb4b7b820d30ef5e319564/roles/0be2075946f54c8485f1c275f746478a` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:46.682037 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c464260d-aae5-43a5-ba57-3c0cfdb74e49 service neutron] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:46.686918 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aebebd-102e-44ba-a080-2226223915fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:46.688446 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aebebd-102e-44ba-a080-2226223915fd admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:46.688854 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aebebd-102e-44ba-a080-2226223915fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:46.689120 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aebebd-102e-44ba-a080-2226223915fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:46.689939 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c408dd-210a-48ab-b006-c0e5086f982f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e093a761d7014c8d81a9a8e3202637a3, user_id=1226c2eb02fb4b7b820d30ef5e319564, role_id=0be2075946f54c8485f1c275f746478a)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:46.690054 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58aebebd-102e-44ba-a080-2226223915fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:46.691596 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58aebebd-102e-44ba-a080-2226223915fd admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:46.692884 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c408dd-210a-48ab-b006-c0e5086f982f admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:46.704213 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 788/1594] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:43:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:43:46.705098 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aebebd-102e-44ba-a080-2226223915fd admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:46.711457 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 807/1595] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:43:46 2022] PUT /identity/v3/projects/e093a761d7014c8d81a9a8e3202637a3/users/1226c2eb02fb4b7b820d30ef5e319564/roles/0be2075946f54c8485f1c275f746478a => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:43:46.712336 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c408dd-210a-48ab-b006-c0e5086f982f admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:46.724183 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aebebd-102e-44ba-a080-2226223915fd admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:46.761946 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:43:46.789924 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c408dd-210a-48ab-b006-c0e5086f982f admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:46.792995 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa837fd-b7d6-4eeb-8c9f-e2f00731fc6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:46.793696 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa837fd-b7d6-4eeb-8c9f-e2f00731fc6a admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:46.794103 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa837fd-b7d6-4eeb-8c9f-e2f00731fc6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:46.794431 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa837fd-b7d6-4eeb-8c9f-e2f00731fc6a admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:46.795036 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa837fd-b7d6-4eeb-8c9f-e2f00731fc6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:46.796253 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7211d752-b858-4efd-a795-b6a372bbf2f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:46.797466 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa837fd-b7d6-4eeb-8c9f-e2f00731fc6a admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:46.797719 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7211d752-b858-4efd-a795-b6a372bbf2f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:46.798459 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7211d752-b858-4efd-a795-b6a372bbf2f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:46.798871 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7211d752-b858-4efd-a795-b6a372bbf2f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:46.799635 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7211d752-b858-4efd-a795-b6a372bbf2f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:46.801362 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7211d752-b858-4efd-a795-b6a372bbf2f4 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:46.808096 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 789/1596] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:43:46 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:43:46.808096 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa837fd-b7d6-4eeb-8c9f-e2f00731fc6a admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:46.819665 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 808/1597] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:43:46 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:43:46.819665 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7211d752-b858-4efd-a795-b6a372bbf2f4 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:46.826710 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa837fd-b7d6-4eeb-8c9f-e2f00731fc6a admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:46.830876 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7211d752-b858-4efd-a795-b6a372bbf2f4 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:46.832954 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc82a547-962d-4192-9f8a-d9b31b22af7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:46.833997 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc82a547-962d-4192-9f8a-d9b31b22af7b admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:46.834375 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc82a547-962d-4192-9f8a-d9b31b22af7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:46.835085 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc82a547-962d-4192-9f8a-d9b31b22af7b admin admin] PATH_INFO: `/v3/projects/375c9f75db854f22a560c9f6408f16b5/users/e577b2f2f18a4be397b6ea6e7f1ba0ad/roles/0be2075946f54c8485f1c275f746478a` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:46.835837 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3f1318-881d-4792-afdb-1b424453591a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:46.836861 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3f1318-881d-4792-afdb-1b424453591a admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:46.837544 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3f1318-881d-4792-afdb-1b424453591a admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:46.837544 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3f1318-881d-4792-afdb-1b424453591a admin admin] PATH_INFO: `/v3/projects/4cf380fb2e904569b24c6212d88ac621/users/8fff53cb0d084d9284825488f6843ee4/roles/0be2075946f54c8485f1c275f746478a` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:46.854035 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc82a547-962d-4192-9f8a-d9b31b22af7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=375c9f75db854f22a560c9f6408f16b5, user_id=e577b2f2f18a4be397b6ea6e7f1ba0ad, role_id=0be2075946f54c8485f1c275f746478a)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:46.856753 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3f1318-881d-4792-afdb-1b424453591a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cf380fb2e904569b24c6212d88ac621, user_id=8fff53cb0d084d9284825488f6843ee4, role_id=0be2075946f54c8485f1c275f746478a)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:46.857215 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc82a547-962d-4192-9f8a-d9b31b22af7b admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:46.867823 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3f1318-881d-4792-afdb-1b424453591a admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:46.887122 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 790/1598] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:43:46 2022] PUT /identity/v3/projects/375c9f75db854f22a560c9f6408f16b5/users/e577b2f2f18a4be397b6ea6e7f1ba0ad/roles/0be2075946f54c8485f1c275f746478a => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:43:46.887399 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc82a547-962d-4192-9f8a-d9b31b22af7b admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:46.894351 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 809/1599] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:43:46 2022] PUT /identity/v3/projects/4cf380fb2e904569b24c6212d88ac621/users/8fff53cb0d084d9284825488f6843ee4/roles/0be2075946f54c8485f1c275f746478a => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:43:46.895204 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3f1318-881d-4792-afdb-1b424453591a admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:46.926949 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:43:46.967633 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc82a547-962d-4192-9f8a-d9b31b22af7b admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:46.974891 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1356d02-f5f7-4e69-b0e8-4839a0761313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:46.975491 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3f1318-881d-4792-afdb-1b424453591a admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:46.975927 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1356d02-f5f7-4e69-b0e8-4839a0761313 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:46.976222 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1356d02-f5f7-4e69-b0e8-4839a0761313 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:46.976701 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1356d02-f5f7-4e69-b0e8-4839a0761313 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:46.977788 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1356d02-f5f7-4e69-b0e8-4839a0761313 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:46.980201 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1356d02-f5f7-4e69-b0e8-4839a0761313 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:46.980778 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c948fa-7ec1-41ed-9c63-f8f61ce075ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:46.982109 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c948fa-7ec1-41ed-9c63-f8f61ce075ce admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:46.982369 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c948fa-7ec1-41ed-9c63-f8f61ce075ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:46.982696 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c948fa-7ec1-41ed-9c63-f8f61ce075ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:46.983397 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c948fa-7ec1-41ed-9c63-f8f61ce075ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:46.985257 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c948fa-7ec1-41ed-9c63-f8f61ce075ce admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:46.994198 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 791/1600] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:43:46 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:43:46.995463 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097f68f3-9d9d-4ed8-93b2-fa7d688f9ef8 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:46.995760 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097f68f3-9d9d-4ed8-93b2-fa7d688f9ef8 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:46.996187 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097f68f3-9d9d-4ed8-93b2-fa7d688f9ef8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:46.996854 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 810/1601] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:43:46 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:43:46.997565 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c948fa-7ec1-41ed-9c63-f8f61ce075ce admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:47.009831 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c948fa-7ec1-41ed-9c63-f8f61ce075ce admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:47.014605 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac97190e-1657-4b2d-b2c0-1673915d4051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:47.016072 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac97190e-1657-4b2d-b2c0-1673915d4051 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:47.016902 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac97190e-1657-4b2d-b2c0-1673915d4051 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:47.016902 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac97190e-1657-4b2d-b2c0-1673915d4051 admin admin] PATH_INFO: `/v3/projects/feb571d68c6c4a1ab9e7b175f5a6e0b3/users/268c5947a265492bb7a8f181f9f5c504/roles/0be2075946f54c8485f1c275f746478a` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:47.033107 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac97190e-1657-4b2d-b2c0-1673915d4051 admin admin] RBAC: Authorizing `identity:create_grant(project_id=feb571d68c6c4a1ab9e7b175f5a6e0b3, user_id=268c5947a265492bb7a8f181f9f5c504, role_id=0be2075946f54c8485f1c275f746478a)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:47.036062 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac97190e-1657-4b2d-b2c0-1673915d4051 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:47.055840 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 811/1602] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:43:46 2022] PUT /identity/v3/projects/feb571d68c6c4a1ab9e7b175f5a6e0b3/users/268c5947a265492bb7a8f181f9f5c504/roles/0be2075946f54c8485f1c275f746478a => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:43:47.055840 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac97190e-1657-4b2d-b2c0-1673915d4051 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:47.059126 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-097f68f3-9d9d-4ed8-93b2-fa7d688f9ef8 None None] MFA Rules not processed for user `1226c2eb02fb4b7b820d30ef5e319564`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:43:47.060363 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-097f68f3-9d9d-4ed8-93b2-fa7d688f9ef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:43:47.144946 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac97190e-1657-4b2d-b2c0-1673915d4051 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:47.153375 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fbd0ad-f93b-4913-827e-1f56a834a970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:47.154283 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fbd0ad-f93b-4913-827e-1f56a834a970 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:47.154557 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fbd0ad-f93b-4913-827e-1f56a834a970 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:47.154831 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fbd0ad-f93b-4913-827e-1f56a834a970 admin admin] PATH_INFO: `/v3/projects/9a7c548d6ba8472fb04ed3999ef86afc/users/68cad55713924deda605b0b9ace133a1/roles/0be2075946f54c8485f1c275f746478a` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:47.157838 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 792/1603] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:43:46 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:43:47.159252 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca383c64-9faa-46cd-a6d3-da875c803354 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:47.160529 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca383c64-9faa-46cd-a6d3-da875c803354 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:47.160883 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca383c64-9faa-46cd-a6d3-da875c803354 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:47.173733 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fbd0ad-f93b-4913-827e-1f56a834a970 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a7c548d6ba8472fb04ed3999ef86afc, user_id=68cad55713924deda605b0b9ace133a1, role_id=0be2075946f54c8485f1c275f746478a)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:47.177487 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fbd0ad-f93b-4913-827e-1f56a834a970 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:47.182033 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-ca383c64-9faa-46cd-a6d3-da875c803354 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:43:47.183379 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-ca383c64-9faa-46cd-a6d3-da875c803354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:43:47.196326 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 793/1604] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:43:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:43:47.196568 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 812/1605] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:43:47 2022] PUT /identity/v3/projects/9a7c548d6ba8472fb04ed3999ef86afc/users/68cad55713924deda605b0b9ace133a1/roles/0be2075946f54c8485f1c275f746478a => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:43:47.196568 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fbd0ad-f93b-4913-827e-1f56a834a970 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:47.196772 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8dc208-b150-492f-b705-b731ee9002bd None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:47.196772 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8dc208-b150-492f-b705-b731ee9002bd None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:47.196772 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8dc208-b150-492f-b705-b731ee9002bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:47.215080 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-1c8dc208-b150-492f-b705-b731ee9002bd None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:43:47.216296 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-1c8dc208-b150-492f-b705-b731ee9002bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:43:47.241340 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:43:47.272989 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fbd0ad-f93b-4913-827e-1f56a834a970 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:47.278582 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdc25cb-4813-4149-a25a-58508a3a214a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:47.280555 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdc25cb-4813-4149-a25a-58508a3a214a admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:47.281530 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdc25cb-4813-4149-a25a-58508a3a214a admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:47.281530 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdc25cb-4813-4149-a25a-58508a3a214a admin admin] PATH_INFO: `/v3/projects/f6aa5a8aecf74b418816e0dfc448388b/users/41141ab13fb04217a351051212a4f8ec/roles/0be2075946f54c8485f1c275f746478a` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:47.293938 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 794/1606] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:43:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:43:47.293938 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8dc208-b150-492f-b705-b731ee9002bd None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:47.301766 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdc25cb-4813-4149-a25a-58508a3a214a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6aa5a8aecf74b418816e0dfc448388b, user_id=41141ab13fb04217a351051212a4f8ec, role_id=0be2075946f54c8485f1c275f746478a)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:47.303750 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8dc208-b150-492f-b705-b731ee9002bd None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:47.304681 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdc25cb-4813-4149-a25a-58508a3a214a admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:47.309461 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5415990-4b00-4795-883c-db5eb19ed34e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:47.310468 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5415990-4b00-4795-883c-db5eb19ed34e admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:47.310855 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5415990-4b00-4795-883c-db5eb19ed34e admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:47.311501 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5415990-4b00-4795-883c-db5eb19ed34e admin admin] PATH_INFO: `/v3/projects/bc6671e3a8bd4821b17ef2d2a426ee58/users/7b078c6bfca04de790e7b2e45739033e/roles/0be2075946f54c8485f1c275f746478a` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:47.314365 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:43:47.323711 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 813/1607] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:43:47 2022] PUT /identity/v3/projects/f6aa5a8aecf74b418816e0dfc448388b/users/41141ab13fb04217a351051212a4f8ec/roles/0be2075946f54c8485f1c275f746478a => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:43:47.325087 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1781aff3-d6be-4394-a325-d8ae0324c11a None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:47.325379 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1781aff3-d6be-4394-a325-d8ae0324c11a None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:47.325672 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1781aff3-d6be-4394-a325-d8ae0324c11a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:47.342349 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5415990-4b00-4795-883c-db5eb19ed34e admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc6671e3a8bd4821b17ef2d2a426ee58, user_id=7b078c6bfca04de790e7b2e45739033e, role_id=0be2075946f54c8485f1c275f746478a)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:47.345853 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5415990-4b00-4795-883c-db5eb19ed34e admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:47.347047 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-1781aff3-d6be-4394-a325-d8ae0324c11a None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:43:47.348577 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-1781aff3-d6be-4394-a325-d8ae0324c11a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:43:47.363487 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 795/1608] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:43:47 2022] PUT /identity/v3/projects/bc6671e3a8bd4821b17ef2d2a426ee58/users/7b078c6bfca04de790e7b2e45739033e/roles/0be2075946f54c8485f1c275f746478a => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:43:47.364315 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5415990-4b00-4795-883c-db5eb19ed34e admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:47.429989 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 814/1609] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:43:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:43:47.431777 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5b9ee5-0a64-42d5-933a-762797bd4cc1 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:47.432140 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5b9ee5-0a64-42d5-933a-762797bd4cc1 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:47.432518 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5b9ee5-0a64-42d5-933a-762797bd4cc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:47.436308 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5415990-4b00-4795-883c-db5eb19ed34e admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:47.443609 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4440301a-fbd1-418a-ad1a-c378232d4775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:47.444408 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4440301a-fbd1-418a-ad1a-c378232d4775 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:47.444769 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4440301a-fbd1-418a-ad1a-c378232d4775 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:47.445543 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4440301a-fbd1-418a-ad1a-c378232d4775 admin admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:47.446455 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4440301a-fbd1-418a-ad1a-c378232d4775 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:47.449053 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4440301a-fbd1-418a-ad1a-c378232d4775 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:47.475012 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-8b5b9ee5-0a64-42d5-933a-762797bd4cc1 None None] MFA Rules not processed for user `68cad55713924deda605b0b9ace133a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:43:47.477972 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-8b5b9ee5-0a64-42d5-933a-762797bd4cc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:43:47.488918 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 796/1610] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jun 16 17:43:47 2022] POST /identity/v3/projects => generated 365 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:43:47.490440 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4440301a-fbd1-418a-ad1a-c378232d4775 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:47.575213 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4440301a-fbd1-418a-ad1a-c378232d4775 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:47.585754 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f64bbb6-1b65-4ace-8766-e210d8eb80bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:47.586518 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f64bbb6-1b65-4ace-8766-e210d8eb80bd service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:47.586875 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f64bbb6-1b65-4ace-8766-e210d8eb80bd service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:47.587227 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f64bbb6-1b65-4ace-8766-e210d8eb80bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:47.596787 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 815/1611] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:43:47 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:43:47.600673 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5b9ee5-0a64-42d5-933a-762797bd4cc1 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:47.649827 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:43:47.680612 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5b9ee5-0a64-42d5-933a-762797bd4cc1 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:47.686373 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032f9e60-6432-49b3-adc0-e233486b1e1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:47.687353 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032f9e60-6432-49b3-adc0-e233486b1e1b service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:47.687645 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032f9e60-6432-49b3-adc0-e233486b1e1b service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:47.687923 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032f9e60-6432-49b3-adc0-e233486b1e1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:47.696596 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f64bbb6-1b65-4ace-8766-e210d8eb80bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:47.698662 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032f9e60-6432-49b3-adc0-e233486b1e1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:47.698883 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f64bbb6-1b65-4ace-8766-e210d8eb80bd service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:47.700131 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032f9e60-6432-49b3-adc0-e233486b1e1b service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:47.716964 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 797/1612] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:43:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:43:47.719527 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befa38b7-1cfd-417f-8074-8385738017f0 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:47.720423 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befa38b7-1cfd-417f-8074-8385738017f0 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:47.720819 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befa38b7-1cfd-417f-8074-8385738017f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:47.723587 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 816/1613] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:43:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:43:47.727270 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed5d04d-0425-463b-ab67-de7efae5c77e None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:47.727573 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed5d04d-0425-463b-ab67-de7efae5c77e None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:47.728126 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed5d04d-0425-463b-ab67-de7efae5c77e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:47.759180 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-befa38b7-1cfd-417f-8074-8385738017f0 None None] MFA Rules not processed for user `41141ab13fb04217a351051212a4f8ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:43:47.760784 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-befa38b7-1cfd-417f-8074-8385738017f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:43:47.763905 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-eed5d04d-0425-463b-ab67-de7efae5c77e None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:43:47.767466 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-eed5d04d-0425-463b-ab67-de7efae5c77e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:43:47.787974 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 817/1614] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:43:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:43:47.791562 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed5d04d-0425-463b-ab67-de7efae5c77e None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:47.817075 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed5d04d-0425-463b-ab67-de7efae5c77e None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:47.825886 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921864b9-baee-44c9-a0ae-64f599923b78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:47.826561 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921864b9-baee-44c9-a0ae-64f599923b78 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:47.826953 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921864b9-baee-44c9-a0ae-64f599923b78 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:47.827243 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921864b9-baee-44c9-a0ae-64f599923b78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:47.839232 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 798/1615] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:43:47 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:43:47.842694 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befa38b7-1cfd-417f-8074-8385738017f0 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:47.852772 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921864b9-baee-44c9-a0ae-64f599923b78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:47.854611 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921864b9-baee-44c9-a0ae-64f599923b78 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:47.856223 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befa38b7-1cfd-417f-8074-8385738017f0 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:47.865551 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef78d39-5da3-4fae-bedb-688c6556d6e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:47.866344 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef78d39-5da3-4fae-bedb-688c6556d6e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:47.867024 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef78d39-5da3-4fae-bedb-688c6556d6e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:47.867024 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef78d39-5da3-4fae-bedb-688c6556d6e2 admin admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:47.867769 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef78d39-5da3-4fae-bedb-688c6556d6e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:47.869936 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef78d39-5da3-4fae-bedb-688c6556d6e2 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:47.873366 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 818/1616] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:43:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:43:47.874535 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921864b9-baee-44c9-a0ae-64f599923b78 service neutron] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:47.898987 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 799/1617] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jun 16 17:43:47 2022] POST /identity/v3/users => generated 289 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:43:47.898987 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef78d39-5da3-4fae-bedb-688c6556d6e2 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:47.900488 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:43:47.909599 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef78d39-5da3-4fae-bedb-688c6556d6e2 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:47.914460 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a9efed-da19-45f9-a9e3-cab3b7b0944f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:47.915168 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a9efed-da19-45f9-a9e3-cab3b7b0944f service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:47.915460 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a9efed-da19-45f9-a9e3-cab3b7b0944f service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:47.915742 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a9efed-da19-45f9-a9e3-cab3b7b0944f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:47.932008 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a9efed-da19-45f9-a9e3-cab3b7b0944f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:47.934419 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a9efed-da19-45f9-a9e3-cab3b7b0944f service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:47.945878 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 800/1618] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:43:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:43:47.947245 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a9efed-da19-45f9-a9e3-cab3b7b0944f service neutron] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:47.957659 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a9efed-da19-45f9-a9e3-cab3b7b0944f service neutron] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:47.964753 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3a924f-42e1-4ecc-9bd6-3772878cbce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:47.965434 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3a924f-42e1-4ecc-9bd6-3772878cbce1 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:47.965885 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3a924f-42e1-4ecc-9bd6-3772878cbce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:47.966298 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3a924f-42e1-4ecc-9bd6-3772878cbce1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:47.967183 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3a924f-42e1-4ecc-9bd6-3772878cbce1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:47.969425 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3a924f-42e1-4ecc-9bd6-3772878cbce1 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:47.975322 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921864b9-baee-44c9-a0ae-64f599923b78 service neutron] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:47.979658 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4468fa59-97ca-43aa-a427-0ad214bc8fae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24d5e2637b7043e08313909d5b06e329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:47.980260 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4468fa59-97ca-43aa-a427-0ad214bc8fae service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:47.980678 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4468fa59-97ca-43aa-a427-0ad214bc8fae service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:47.981055 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4468fa59-97ca-43aa-a427-0ad214bc8fae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:48.018127 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 801/1619] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jun 16 17:43:47 2022] POST /identity/v3/projects => generated 405 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:43:48.019288 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3a924f-42e1-4ecc-9bd6-3772878cbce1 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:48.100533 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4468fa59-97ca-43aa-a427-0ad214bc8fae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:48.103412 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4468fa59-97ca-43aa-a427-0ad214bc8fae service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:48.116705 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 819/1620] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:43:47 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:43:48.117392 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4468fa59-97ca-43aa-a427-0ad214bc8fae service cinder] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:48.123129 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3a924f-42e1-4ecc-9bd6-3772878cbce1 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:48.131870 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4468fa59-97ca-43aa-a427-0ad214bc8fae service cinder] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:48.137040 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b9e76f-733f-4737-9624-a38e423b10aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:48.138273 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b9e76f-733f-4737-9624-a38e423b10aa admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:48.139042 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c1c684-7102-4947-ab27-16954fbd547c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:48.139182 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b9e76f-733f-4737-9624-a38e423b10aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:48.139278 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c1c684-7102-4947-ab27-16954fbd547c admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:48.139495 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c1c684-7102-4947-ab27-16954fbd547c admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:48.139777 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c1c684-7102-4947-ab27-16954fbd547c admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:48.139888 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b9e76f-733f-4737-9624-a38e423b10aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:48.140721 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c1c684-7102-4947-ab27-16954fbd547c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:48.140960 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b9e76f-733f-4737-9624-a38e423b10aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:48.143480 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b9e76f-733f-4737-9624-a38e423b10aa admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:48.149186 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c1c684-7102-4947-ab27-16954fbd547c admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:48.160475 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 802/1621] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:43:48 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:43:48.162080 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b9e76f-733f-4737-9624-a38e423b10aa admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:48.178694 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b9e76f-733f-4737-9624-a38e423b10aa admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:48.184704 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95103b1-b115-4d5b-a5b0-ed473e42f17e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:48.185766 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95103b1-b115-4d5b-a5b0-ed473e42f17e admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:48.186800 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95103b1-b115-4d5b-a5b0-ed473e42f17e admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:48.188025 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95103b1-b115-4d5b-a5b0-ed473e42f17e admin admin] PATH_INFO: `/v3/projects/96f4e5ab7403467bbe419ea024009a58/users/a9a1d23a442b43929b9cf03bdc0e6be6/roles/3ca2dcbbd29d4a31a3ab78f2299bc239` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:48.205465 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 820/1622] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jun 16 17:43:48 2022] POST /identity/v3/users => generated 309 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:43:48.207461 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:43:48.208841 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c1c684-7102-4947-ab27-16954fbd547c admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:48.219710 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95103b1-b115-4d5b-a5b0-ed473e42f17e admin admin] RBAC: Authorizing `identity:create_grant(project_id=96f4e5ab7403467bbe419ea024009a58, user_id=a9a1d23a442b43929b9cf03bdc0e6be6, role_id=3ca2dcbbd29d4a31a3ab78f2299bc239)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:48.223330 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95103b1-b115-4d5b-a5b0-ed473e42f17e admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:48.236479 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c1c684-7102-4947-ab27-16954fbd547c admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:48.244630 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 803/1623] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:43:48 2022] PUT /identity/v3/projects/96f4e5ab7403467bbe419ea024009a58/users/a9a1d23a442b43929b9cf03bdc0e6be6/roles/3ca2dcbbd29d4a31a3ab78f2299bc239 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:43:48.248790 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95103b1-b115-4d5b-a5b0-ed473e42f17e admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:48.406527 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd18145c-56ff-4e8d-9607-fc8de7ea7abb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:48.406527 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd18145c-56ff-4e8d-9607-fc8de7ea7abb admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:48.409026 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95103b1-b115-4d5b-a5b0-ed473e42f17e admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:48.409519 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd18145c-56ff-4e8d-9607-fc8de7ea7abb admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:48.409721 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd18145c-56ff-4e8d-9607-fc8de7ea7abb admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:48.410720 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd18145c-56ff-4e8d-9607-fc8de7ea7abb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:48.414236 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd18145c-56ff-4e8d-9607-fc8de7ea7abb admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:48.427278 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7094b4e0-80a2-452b-a3f8-4898268dec4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:48.427872 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7094b4e0-80a2-452b-a3f8-4898268dec4a admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:48.428772 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7094b4e0-80a2-452b-a3f8-4898268dec4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:48.429181 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7094b4e0-80a2-452b-a3f8-4898268dec4a admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:48.432626 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7094b4e0-80a2-452b-a3f8-4898268dec4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:48.438479 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7094b4e0-80a2-452b-a3f8-4898268dec4a admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:48.448455 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 821/1624] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:43:48 2022] GET /identity/v3/roles => generated 1420 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:43:48.450621 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd18145c-56ff-4e8d-9607-fc8de7ea7abb admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:48.465857 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 804/1625] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:43:48 2022] GET /identity/v3/roles => generated 1420 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:43:48.472184 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7094b4e0-80a2-452b-a3f8-4898268dec4a admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:48.478855 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd18145c-56ff-4e8d-9607-fc8de7ea7abb admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:48.494190 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3f23e4-a034-426d-b8a8-a989ccaeee0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:48.499022 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3f23e4-a034-426d-b8a8-a989ccaeee0f admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:48.499452 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3f23e4-a034-426d-b8a8-a989ccaeee0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:48.500414 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3f23e4-a034-426d-b8a8-a989ccaeee0f admin admin] PATH_INFO: `/v3/projects/6d3aad1197d343819eacd03b649f877f/users/41c34f560f454030af45b2d1e1fb46ab/roles/3ca2dcbbd29d4a31a3ab78f2299bc239` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:48.500699 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7094b4e0-80a2-452b-a3f8-4898268dec4a admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:48.516990 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85f52da-25c1-465e-8735-4e42bd67dab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:48.518218 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85f52da-25c1-465e-8735-4e42bd67dab9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:48.518741 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85f52da-25c1-465e-8735-4e42bd67dab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:48.519121 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85f52da-25c1-465e-8735-4e42bd67dab9 admin admin] PATH_INFO: `/v3/projects/96f4e5ab7403467bbe419ea024009a58/users/a9a1d23a442b43929b9cf03bdc0e6be6/roles/3ca2dcbbd29d4a31a3ab78f2299bc239` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:48.523015 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85f52da-25c1-465e-8735-4e42bd67dab9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=96f4e5ab7403467bbe419ea024009a58, user_id=a9a1d23a442b43929b9cf03bdc0e6be6, role_id=3ca2dcbbd29d4a31a3ab78f2299bc239)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:48.528343 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85f52da-25c1-465e-8735-4e42bd67dab9 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:48.536346 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3f23e4-a034-426d-b8a8-a989ccaeee0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d3aad1197d343819eacd03b649f877f, user_id=41c34f560f454030af45b2d1e1fb46ab, role_id=3ca2dcbbd29d4a31a3ab78f2299bc239)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:48.542411 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3f23e4-a034-426d-b8a8-a989ccaeee0f admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:48.551444 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 805/1626] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:43:48 2022] PUT /identity/v3/projects/96f4e5ab7403467bbe419ea024009a58/users/a9a1d23a442b43929b9cf03bdc0e6be6/roles/3ca2dcbbd29d4a31a3ab78f2299bc239 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:43:48.564047 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef9c60d-2a4a-4268-9ce4-e3e3e922038f None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:48.564486 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef9c60d-2a4a-4268-9ce4-e3e3e922038f None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:48.564828 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef9c60d-2a4a-4268-9ce4-e3e3e922038f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:48.574441 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 822/1627] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:43:48 2022] PUT /identity/v3/projects/6d3aad1197d343819eacd03b649f877f/users/41c34f560f454030af45b2d1e1fb46ab/roles/3ca2dcbbd29d4a31a3ab78f2299bc239 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:43:48.586975 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3f23e4-a034-426d-b8a8-a989ccaeee0f admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:48.644093 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-eef9c60d-2a4a-4268-9ce4-e3e3e922038f None None] MFA Rules not processed for user `a9a1d23a442b43929b9cf03bdc0e6be6`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:43:48.645393 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-eef9c60d-2a4a-4268-9ce4-e3e3e922038f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:43:48.652556 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:43:48.822484 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3f23e4-a034-426d-b8a8-a989ccaeee0f admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:48.838475 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db727da-5192-4095-84ad-ef4f0361b1df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:48.839132 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db727da-5192-4095-84ad-ef4f0361b1df admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:48.841150 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db727da-5192-4095-84ad-ef4f0361b1df admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:48.841438 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db727da-5192-4095-84ad-ef4f0361b1df admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:48.843389 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db727da-5192-4095-84ad-ef4f0361b1df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:48.848104 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db727da-5192-4095-84ad-ef4f0361b1df admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:48.876417 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 823/1628] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:43:48 2022] GET /identity/v3/roles => generated 1420 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:43:48.898325 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db727da-5192-4095-84ad-ef4f0361b1df admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:48.912115 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 806/1629] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:43:48 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 349 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:43:48.919984 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db727da-5192-4095-84ad-ef4f0361b1df admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:48.936808 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1470b543-eaee-44bf-a3c4-7d390254725f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:48.939789 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1470b543-eaee-44bf-a3c4-7d390254725f admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:48.940556 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1470b543-eaee-44bf-a3c4-7d390254725f admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:48.940842 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1470b543-eaee-44bf-a3c4-7d390254725f admin admin] PATH_INFO: `/v3/projects/6d3aad1197d343819eacd03b649f877f/users/41c34f560f454030af45b2d1e1fb46ab/roles/3ca2dcbbd29d4a31a3ab78f2299bc239` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:48.945245 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1470b543-eaee-44bf-a3c4-7d390254725f admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d3aad1197d343819eacd03b649f877f, user_id=41c34f560f454030af45b2d1e1fb46ab, role_id=3ca2dcbbd29d4a31a3ab78f2299bc239)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:48.955138 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef9c60d-2a4a-4268-9ce4-e3e3e922038f None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:48.974886 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1470b543-eaee-44bf-a3c4-7d390254725f admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:49.020794 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 824/1630] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:43:48 2022] PUT /identity/v3/projects/6d3aad1197d343819eacd03b649f877f/users/41c34f560f454030af45b2d1e1fb46ab/roles/3ca2dcbbd29d4a31a3ab78f2299bc239 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:43:49.037038 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d767e54-610e-4ca8-b07e-78c5a3a0d30b None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:49.038211 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d767e54-610e-4ca8-b07e-78c5a3a0d30b None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:49.038880 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d767e54-610e-4ca8-b07e-78c5a3a0d30b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:49.122036 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef9c60d-2a4a-4268-9ce4-e3e3e922038f None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:49.126677 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-4d767e54-610e-4ca8-b07e-78c5a3a0d30b None None] MFA Rules not processed for user `41c34f560f454030af45b2d1e1fb46ab`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:43:49.128264 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-4d767e54-610e-4ca8-b07e-78c5a3a0d30b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:43:49.137825 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:43:49.228389 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdce7b7a-9da0-41a1-acc7-953d327b1980 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24d5e2637b7043e08313909d5b06e329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:49.229179 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdce7b7a-9da0-41a1-acc7-953d327b1980 service cinder] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:49.229822 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdce7b7a-9da0-41a1-acc7-953d327b1980 service cinder] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:49.229822 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdce7b7a-9da0-41a1-acc7-953d327b1980 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:49.292321 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 825/1631] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:43:49 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:43:49.306789 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d767e54-610e-4ca8-b07e-78c5a3a0d30b None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:49.320002 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d767e54-610e-4ca8-b07e-78c5a3a0d30b None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:49.327651 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e37fb5-3d40-4947-a655-cea5f0535d40 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24d5e2637b7043e08313909d5b06e329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:49.329571 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e37fb5-3d40-4947-a655-cea5f0535d40 service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:49.329728 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e37fb5-3d40-4947-a655-cea5f0535d40 service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:49.329901 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e37fb5-3d40-4947-a655-cea5f0535d40 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:49.341788 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e37fb5-3d40-4947-a655-cea5f0535d40 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:49.344061 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdce7b7a-9da0-41a1-acc7-953d327b1980 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:49.345816 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdce7b7a-9da0-41a1-acc7-953d327b1980 service cinder] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:49.350436 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e37fb5-3d40-4947-a655-cea5f0535d40 service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:49.369698 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 826/1632] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:43:49 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:43:49.372695 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 807/1633] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:43:48 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 17:43:49.646992 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e37fb5-3d40-4947-a655-cea5f0535d40 service cinder] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:49.675766 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e37fb5-3d40-4947-a655-cea5f0535d40 service cinder] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:49.683021 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6c34db-3676-4996-bc69-34f9a87215e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24d5e2637b7043e08313909d5b06e329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:49.683788 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6c34db-3676-4996-bc69-34f9a87215e8 service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:49.684080 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6c34db-3676-4996-bc69-34f9a87215e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:49.684362 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6c34db-3676-4996-bc69-34f9a87215e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:49.805068 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6c34db-3676-4996-bc69-34f9a87215e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:49.808744 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6c34db-3676-4996-bc69-34f9a87215e8 service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:49.839833 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 827/1634] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:43:49 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:43:49.950660 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdce7b7a-9da0-41a1-acc7-953d327b1980 service cinder] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:50.116717 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdce7b7a-9da0-41a1-acc7-953d327b1980 service cinder] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:50.145031 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd55d779-26a3-46b2-8fd9-a036e0cb34f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:50.150756 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd55d779-26a3-46b2-8fd9-a036e0cb34f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:50.151647 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd55d779-26a3-46b2-8fd9-a036e0cb34f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:50.152320 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd55d779-26a3-46b2-8fd9-a036e0cb34f4 admin admin] PATH_INFO: `/v3/users/41141ab13fb04217a351051212a4f8ec` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:50.154435 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd55d779-26a3-46b2-8fd9-a036e0cb34f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41141ab13fb04217a351051212a4f8ec)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:50.164663 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: WARNING py.warnings [None req-cd55d779-26a3-46b2-8fd9-a036e0cb34f4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 17:43:50.164663 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: warnings.warn(msg) Jun 16 17:43:50.164663 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: Jun 16 17:43:50.166378 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd55d779-26a3-46b2-8fd9-a036e0cb34f4 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:50.324064 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-cd55d779-26a3-46b2-8fd9-a036e0cb34f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41141ab13fb04217a351051212a4f8ec'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:43:50.324815 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-cd55d779-26a3-46b2-8fd9-a036e0cb34f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41141ab13fb04217a351051212a4f8ec'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:43:50.352107 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-cd55d779-26a3-46b2-8fd9-a036e0cb34f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41141ab13fb04217a351051212a4f8ec'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:43:50.352747 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-cd55d779-26a3-46b2-8fd9-a036e0cb34f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41141ab13fb04217a351051212a4f8ec'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:43:50.376788 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:43:50.390768 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-cd55d779-26a3-46b2-8fd9-a036e0cb34f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41141ab13fb04217a351051212a4f8ec'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:43:50.432106 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 808/1635] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jun 16 17:43:49 2022] DELETE /identity/v3/users/41141ab13fb04217a351051212a4f8ec => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:43:50.439364 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbacbac-dd80-4e20-aa7c-5f42aab5cf1b None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:50.439853 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbacbac-dd80-4e20-aa7c-5f42aab5cf1b None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:50.441089 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbacbac-dd80-4e20-aa7c-5f42aab5cf1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:50.486849 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-2fbacbac-dd80-4e20-aa7c-5f42aab5cf1b None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:43:50.502886 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-2fbacbac-dd80-4e20-aa7c-5f42aab5cf1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:43:50.539016 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:43:50.763011 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 828/1636] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:43:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:43:50.807019 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd55d779-26a3-46b2-8fd9-a036e0cb34f4 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:50.809110 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-cd55d779-26a3-46b2-8fd9-a036e0cb34f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:43:50.947672 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd55d779-26a3-46b2-8fd9-a036e0cb34f4 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:50.970033 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bd1e12-4cc6-4036-98fe-4070d38439cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:50.972413 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bd1e12-4cc6-4036-98fe-4070d38439cf service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:50.972729 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bd1e12-4cc6-4036-98fe-4070d38439cf service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:50.973019 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bd1e12-4cc6-4036-98fe-4070d38439cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:51.001191 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bd1e12-4cc6-4036-98fe-4070d38439cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:51.002953 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bd1e12-4cc6-4036-98fe-4070d38439cf service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:51.027559 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 809/1637] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:43:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:43:51.049588 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fffcc9-bcbf-416a-8cb9-3b54ba37cd3b None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:51.050283 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fffcc9-bcbf-416a-8cb9-3b54ba37cd3b None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:51.051393 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fffcc9-bcbf-416a-8cb9-3b54ba37cd3b None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:51.053441 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 829/1638] 10.222.0.42 () {54 vars in 961 bytes} [Thu Jun 16 17:43:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:43:51.062113 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2f8671-e446-4571-86da-0291a23a51c3 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:51.063490 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2f8671-e446-4571-86da-0291a23a51c3 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:51.064500 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2f8671-e446-4571-86da-0291a23a51c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:51.096772 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-5c2f8671-e446-4571-86da-0291a23a51c3 None None] MFA Rules not processed for user `f86c16179a70474a817be9fda33ece9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:43:51.098262 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-5c2f8671-e446-4571-86da-0291a23a51c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:43:51.213420 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 810/1639] 10.222.0.42 () {58 vars in 1058 bytes} [Thu Jun 16 17:43:51 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:43:51.214738 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f7bbbc-5c22-4803-b1a6-5f4ba40b6358 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:51.215292 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f7bbbc-5c22-4803-b1a6-5f4ba40b6358 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:51.215586 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f7bbbc-5c22-4803-b1a6-5f4ba40b6358 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:51.217346 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 830/1640] 10.222.0.42 () {54 vars in 912 bytes} [Thu Jun 16 17:43:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:43:51.230551 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580a5173-a87f-4a39-b185-2c650764bbdf None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:51.233457 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580a5173-a87f-4a39-b185-2c650764bbdf None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:51.235548 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580a5173-a87f-4a39-b185-2c650764bbdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:51.271379 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-580a5173-a87f-4a39-b185-2c650764bbdf None None] MFA Rules not processed for user `f86c16179a70474a817be9fda33ece9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:43:51.273491 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-580a5173-a87f-4a39-b185-2c650764bbdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:43:51.299749 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 811/1641] 10.222.0.42 () {58 vars in 1009 bytes} [Thu Jun 16 17:43:51 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:43:51.307746 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f7bbbc-5c22-4803-b1a6-5f4ba40b6358 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:51.309668 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-d2f7bbbc-5c22-4803-b1a6-5f4ba40b6358 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:43:51.405482 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f7bbbc-5c22-4803-b1a6-5f4ba40b6358 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:51.411588 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fefb2b-7f47-40d3-8742-551afa1f3372 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24d5e2637b7043e08313909d5b06e329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:51.412554 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fefb2b-7f47-40d3-8742-551afa1f3372 service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:51.413743 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fefb2b-7f47-40d3-8742-551afa1f3372 service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:51.413743 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fefb2b-7f47-40d3-8742-551afa1f3372 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:51.436132 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fefb2b-7f47-40d3-8742-551afa1f3372 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:51.439084 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fefb2b-7f47-40d3-8742-551afa1f3372 service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:51.458634 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 831/1642] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:43:51 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:43:52.014848 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580a5173-a87f-4a39-b185-2c650764bbdf None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:52.016884 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-580a5173-a87f-4a39-b185-2c650764bbdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:43:52.042063 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580a5173-a87f-4a39-b185-2c650764bbdf None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:52.051524 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da732a5-1bba-490f-8fa8-ba22d3a4377f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:52.053235 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da732a5-1bba-490f-8fa8-ba22d3a4377f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:52.054267 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da732a5-1bba-490f-8fa8-ba22d3a4377f admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:52.056218 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da732a5-1bba-490f-8fa8-ba22d3a4377f admin admin] PATH_INFO: `/v3/projects/f6aa5a8aecf74b418816e0dfc448388b` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:52.060010 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da732a5-1bba-490f-8fa8-ba22d3a4377f admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6aa5a8aecf74b418816e0dfc448388b)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:52.064844 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da732a5-1bba-490f-8fa8-ba22d3a4377f admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:52.177645 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-2da732a5-1bba-490f-8fa8-ba22d3a4377f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6aa5a8aecf74b418816e0dfc448388b', 'request_id': 'req-2da732a5-1bba-490f-8fa8-ba22d3a4377f'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:43:52.203465 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-2da732a5-1bba-490f-8fa8-ba22d3a4377f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6aa5a8aecf74b418816e0dfc448388b', 'request_id': 'req-2da732a5-1bba-490f-8fa8-ba22d3a4377f'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:43:52.210500 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:43:52.212580 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-2da732a5-1bba-490f-8fa8-ba22d3a4377f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6aa5a8aecf74b418816e0dfc448388b', 'request_id': 'req-2da732a5-1bba-490f-8fa8-ba22d3a4377f'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:43:52.238438 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-2da732a5-1bba-490f-8fa8-ba22d3a4377f admin admin] The token cache is being invalidate because project f6aa5a8aecf74b418816e0dfc448388b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80357) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:43:52.239059 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-2da732a5-1bba-490f-8fa8-ba22d3a4377f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:43:52.241187 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-2da732a5-1bba-490f-8fa8-ba22d3a4377f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:43:52.247803 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 812/1643] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jun 16 17:43:52 2022] DELETE /identity/v3/projects/f6aa5a8aecf74b418816e0dfc448388b => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:43:52.253265 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fefb2b-7f47-40d3-8742-551afa1f3372 service cinder] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:52.274375 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fefb2b-7f47-40d3-8742-551afa1f3372 service cinder] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:52.522117 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3de3e02-f61c-4f32-91ae-246a12452547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:52.526642 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3de3e02-f61c-4f32-91ae-246a12452547 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:52.527739 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3de3e02-f61c-4f32-91ae-246a12452547 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:52.528424 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3de3e02-f61c-4f32-91ae-246a12452547 admin admin] PATH_INFO: `/v3/users/41c34f560f454030af45b2d1e1fb46ab` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:52.530686 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3de3e02-f61c-4f32-91ae-246a12452547 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41c34f560f454030af45b2d1e1fb46ab)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:52.537589 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3de3e02-f61c-4f32-91ae-246a12452547 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:52.649116 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-a3de3e02-f61c-4f32-91ae-246a12452547 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41c34f560f454030af45b2d1e1fb46ab'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:43:52.677175 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:43:52.687145 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-a3de3e02-f61c-4f32-91ae-246a12452547 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41c34f560f454030af45b2d1e1fb46ab'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:43:52.690405 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-a3de3e02-f61c-4f32-91ae-246a12452547 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41c34f560f454030af45b2d1e1fb46ab'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:43:52.710079 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-a3de3e02-f61c-4f32-91ae-246a12452547 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41c34f560f454030af45b2d1e1fb46ab'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:43:52.727799 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-a3de3e02-f61c-4f32-91ae-246a12452547 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41c34f560f454030af45b2d1e1fb46ab'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:43:52.742871 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 832/1644] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jun 16 17:43:52 2022] DELETE /identity/v3/users/41c34f560f454030af45b2d1e1fb46ab => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:43:53.965480 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d80b4cc-ed63-4e92-ad4a-f672548cd185 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:53.966068 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d80b4cc-ed63-4e92-ad4a-f672548cd185 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:53.966068 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d80b4cc-ed63-4e92-ad4a-f672548cd185 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:53.973104 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 813/1645] 10.222.0.42 () {54 vars in 961 bytes} [Thu Jun 16 17:43:53 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:43:53.993371 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074a397f-b06e-4b8d-a058-e1f05c7d5781 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:53.993371 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074a397f-b06e-4b8d-a058-e1f05c7d5781 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:53.994567 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074a397f-b06e-4b8d-a058-e1f05c7d5781 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:54.025135 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-074a397f-b06e-4b8d-a058-e1f05c7d5781 None None] MFA Rules not processed for user `f86c16179a70474a817be9fda33ece9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:43:54.027671 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-074a397f-b06e-4b8d-a058-e1f05c7d5781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:43:54.099508 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d80b4cc-ed63-4e92-ad4a-f672548cd185 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:54.104284 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-4d80b4cc-ed63-4e92-ad4a-f672548cd185 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:43:54.136067 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 833/1646] 10.222.0.42 () {58 vars in 1058 bytes} [Thu Jun 16 17:43:53 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:43:54.137562 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb5e536-4a20-47d0-92c8-dbff8db76f78 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:54.138446 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb5e536-4a20-47d0-92c8-dbff8db76f78 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:54.138792 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb5e536-4a20-47d0-92c8-dbff8db76f78 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:54.141065 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 834/1647] 10.222.0.42 () {54 vars in 912 bytes} [Thu Jun 16 17:43:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:43:54.146758 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e935175-0dcf-48e5-bca9-1e69bb395007 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:54.147040 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e935175-0dcf-48e5-bca9-1e69bb395007 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:54.147318 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e935175-0dcf-48e5-bca9-1e69bb395007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:54.184418 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-7e935175-0dcf-48e5-bca9-1e69bb395007 None None] MFA Rules not processed for user `f86c16179a70474a817be9fda33ece9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:43:54.185921 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-7e935175-0dcf-48e5-bca9-1e69bb395007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:43:54.232034 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 835/1648] 10.222.0.42 () {58 vars in 1009 bytes} [Thu Jun 16 17:43:54 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:43:54.248186 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e935175-0dcf-48e5-bca9-1e69bb395007 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:54.251470 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-7e935175-0dcf-48e5-bca9-1e69bb395007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:43:54.307033 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d80b4cc-ed63-4e92-ad4a-f672548cd185 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:54.319631 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebbc8f0-9e12-44fb-b612-bcdda8c56b2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:54.320721 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebbc8f0-9e12-44fb-b612-bcdda8c56b2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:54.321046 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebbc8f0-9e12-44fb-b612-bcdda8c56b2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:54.321323 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebbc8f0-9e12-44fb-b612-bcdda8c56b2e admin admin] PATH_INFO: `/v3/users/1226c2eb02fb4b7b820d30ef5e319564` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:54.323226 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebbc8f0-9e12-44fb-b612-bcdda8c56b2e admin admin] RBAC: Authorizing `identity:delete_user(user_id=1226c2eb02fb4b7b820d30ef5e319564)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:54.325514 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebbc8f0-9e12-44fb-b612-bcdda8c56b2e admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:54.392171 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e935175-0dcf-48e5-bca9-1e69bb395007 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:54.402231 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60aa4d6e-d6d0-4267-ace6-b2981dad403a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24d5e2637b7043e08313909d5b06e329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:54.404729 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aa4d6e-d6d0-4267-ace6-b2981dad403a service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:54.406364 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aa4d6e-d6d0-4267-ace6-b2981dad403a service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:54.407659 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aa4d6e-d6d0-4267-ace6-b2981dad403a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:54.425158 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60aa4d6e-d6d0-4267-ace6-b2981dad403a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:54.427326 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60aa4d6e-d6d0-4267-ace6-b2981dad403a service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:54.461831 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-5ebbc8f0-9e12-44fb-b612-bcdda8c56b2e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1226c2eb02fb4b7b820d30ef5e319564'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:43:54.462547 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-5ebbc8f0-9e12-44fb-b612-bcdda8c56b2e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1226c2eb02fb4b7b820d30ef5e319564'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:43:54.472540 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 836/1649] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:43:54 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 17:43:54.474878 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-5ebbc8f0-9e12-44fb-b612-bcdda8c56b2e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1226c2eb02fb4b7b820d30ef5e319564'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:43:54.475527 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-5ebbc8f0-9e12-44fb-b612-bcdda8c56b2e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1226c2eb02fb4b7b820d30ef5e319564'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:43:54.497439 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-5ebbc8f0-9e12-44fb-b612-bcdda8c56b2e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1226c2eb02fb4b7b820d30ef5e319564'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:43:54.530306 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 814/1650] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jun 16 17:43:54 2022] DELETE /identity/v3/users/1226c2eb02fb4b7b820d30ef5e319564 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:43:54.535730 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef507d71-70ec-41fb-8d85-3bf4b8df2ef1 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:54.536255 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef507d71-70ec-41fb-8d85-3bf4b8df2ef1 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:54.536664 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef507d71-70ec-41fb-8d85-3bf4b8df2ef1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:54.583528 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-ef507d71-70ec-41fb-8d85-3bf4b8df2ef1 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:43:54.595011 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-ef507d71-70ec-41fb-8d85-3bf4b8df2ef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:43:54.741273 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 837/1651] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:43:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:43:54.784669 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebbc8f0-9e12-44fb-b612-bcdda8c56b2e admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:54.786470 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-5ebbc8f0-9e12-44fb-b612-bcdda8c56b2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:43:54.890638 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebbc8f0-9e12-44fb-b612-bcdda8c56b2e admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:54.896165 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65862559-409e-4a25-8da2-d1afd86cb123 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:54.897571 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65862559-409e-4a25-8da2-d1afd86cb123 service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:54.899487 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65862559-409e-4a25-8da2-d1afd86cb123 service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:54.899865 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65862559-409e-4a25-8da2-d1afd86cb123 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:54.921267 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65862559-409e-4a25-8da2-d1afd86cb123 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:54.927364 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65862559-409e-4a25-8da2-d1afd86cb123 service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:54.959181 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 815/1652] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:43:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:43:55.151766 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef507d71-70ec-41fb-8d85-3bf4b8df2ef1 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:55.153628 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-ef507d71-70ec-41fb-8d85-3bf4b8df2ef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:43:55.170853 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:43:55.184841 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef507d71-70ec-41fb-8d85-3bf4b8df2ef1 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:55.195848 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d26563-7bcb-4196-8880-f74e253eb527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:55.206162 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d26563-7bcb-4196-8880-f74e253eb527 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:55.206824 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d26563-7bcb-4196-8880-f74e253eb527 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:55.207307 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d26563-7bcb-4196-8880-f74e253eb527 admin admin] PATH_INFO: `/v3/projects/6d3aad1197d343819eacd03b649f877f` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:55.209555 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d26563-7bcb-4196-8880-f74e253eb527 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d3aad1197d343819eacd03b649f877f)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:55.212788 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: WARNING py.warnings [None req-e6d26563-7bcb-4196-8880-f74e253eb527 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 17:43:55.212788 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: warnings.warn(msg) Jun 16 17:43:55.212788 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: Jun 16 17:43:55.215431 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d26563-7bcb-4196-8880-f74e253eb527 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:55.639975 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-e6d26563-7bcb-4196-8880-f74e253eb527 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d3aad1197d343819eacd03b649f877f', 'request_id': 'req-e6d26563-7bcb-4196-8880-f74e253eb527'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:43:55.660233 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-e6d26563-7bcb-4196-8880-f74e253eb527 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d3aad1197d343819eacd03b649f877f', 'request_id': 'req-e6d26563-7bcb-4196-8880-f74e253eb527'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:43:55.684311 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-e6d26563-7bcb-4196-8880-f74e253eb527 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d3aad1197d343819eacd03b649f877f', 'request_id': 'req-e6d26563-7bcb-4196-8880-f74e253eb527'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:43:55.710850 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-e6d26563-7bcb-4196-8880-f74e253eb527 admin admin] The token cache is being invalidate because project 6d3aad1197d343819eacd03b649f877f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:43:55.711165 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-e6d26563-7bcb-4196-8880-f74e253eb527 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:43:55.713035 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-e6d26563-7bcb-4196-8880-f74e253eb527 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:43:55.719091 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 838/1653] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jun 16 17:43:55 2022] DELETE /identity/v3/projects/6d3aad1197d343819eacd03b649f877f => generated 0 bytes in 567 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:43:55.723644 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755ebade-80d9-4b48-99d7-000310f19deb None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:55.724093 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755ebade-80d9-4b48-99d7-000310f19deb None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:55.724746 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755ebade-80d9-4b48-99d7-000310f19deb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:55.746715 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-755ebade-80d9-4b48-99d7-000310f19deb None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:43:55.748303 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-755ebade-80d9-4b48-99d7-000310f19deb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:43:55.762021 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:43:55.866898 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 816/1654] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:43:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:43:55.875798 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6138bbc7-a0ee-48fa-b927-94ae5e850546 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:55.876233 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6138bbc7-a0ee-48fa-b927-94ae5e850546 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:55.877009 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6138bbc7-a0ee-48fa-b927-94ae5e850546 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:55.900939 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-6138bbc7-a0ee-48fa-b927-94ae5e850546 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:43:55.902556 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-6138bbc7-a0ee-48fa-b927-94ae5e850546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:43:55.913270 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 839/1655] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:43:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:43:55.917559 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755ebade-80d9-4b48-99d7-000310f19deb None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:55.930658 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755ebade-80d9-4b48-99d7-000310f19deb None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:55.949569 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf31cef-c979-40e3-9da3-0f75b1207e4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:55.950339 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf31cef-c979-40e3-9da3-0f75b1207e4b admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:55.950541 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf31cef-c979-40e3-9da3-0f75b1207e4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:55.950818 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf31cef-c979-40e3-9da3-0f75b1207e4b admin admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:55.951563 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf31cef-c979-40e3-9da3-0f75b1207e4b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:43:55.952022 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf31cef-c979-40e3-9da3-0f75b1207e4b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:55.955427 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf31cef-c979-40e3-9da3-0f75b1207e4b admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:55.972643 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 817/1656] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jun 16 17:43:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:43:55.979376 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6138bbc7-a0ee-48fa-b927-94ae5e850546 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:55.991487 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6138bbc7-a0ee-48fa-b927-94ae5e850546 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:55.998650 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5546762-9f32-4034-b866-be95f069cce4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:56.000090 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5546762-9f32-4034-b866-be95f069cce4 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:56.001263 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5546762-9f32-4034-b866-be95f069cce4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:56.001692 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5546762-9f32-4034-b866-be95f069cce4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:56.002879 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5546762-9f32-4034-b866-be95f069cce4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:56.006054 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5546762-9f32-4034-b866-be95f069cce4 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:56.048115 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 840/1657] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jun 16 17:43:55 2022] POST /identity/v3/projects => generated 377 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:43:56.052415 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf31cef-c979-40e3-9da3-0f75b1207e4b admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:56.070457 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf31cef-c979-40e3-9da3-0f75b1207e4b admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:56.173313 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2c11ca-6290-41c5-910c-71555ca89f32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:56.174711 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2c11ca-6290-41c5-910c-71555ca89f32 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:56.175015 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2c11ca-6290-41c5-910c-71555ca89f32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:56.175334 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2c11ca-6290-41c5-910c-71555ca89f32 admin admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:56.176068 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2c11ca-6290-41c5-910c-71555ca89f32 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:56.179681 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2c11ca-6290-41c5-910c-71555ca89f32 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:56.222768 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 818/1658] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jun 16 17:43:56 2022] POST /identity/v3/users => generated 289 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:43:56.228971 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5546762-9f32-4034-b866-be95f069cce4 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:56.250750 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5546762-9f32-4034-b866-be95f069cce4 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:56.266650 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5567614e-8af2-4a4f-959e-d543d239f27b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:56.267306 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5567614e-8af2-4a4f-959e-d543d239f27b admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:56.267668 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5567614e-8af2-4a4f-959e-d543d239f27b admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:56.267834 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5567614e-8af2-4a4f-959e-d543d239f27b admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:56.268602 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5567614e-8af2-4a4f-959e-d543d239f27b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:56.270268 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5567614e-8af2-4a4f-959e-d543d239f27b admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:56.292222 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 841/1659] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:43:56 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:43:56.304787 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2c11ca-6290-41c5-910c-71555ca89f32 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:56.361988 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2c11ca-6290-41c5-910c-71555ca89f32 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:56.370206 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4bd7b1-b3aa-438f-a067-44fd13e638fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:56.370983 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4bd7b1-b3aa-438f-a067-44fd13e638fb admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:56.371336 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4bd7b1-b3aa-438f-a067-44fd13e638fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:56.371685 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4bd7b1-b3aa-438f-a067-44fd13e638fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:56.372662 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4bd7b1-b3aa-438f-a067-44fd13e638fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:56.374924 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4bd7b1-b3aa-438f-a067-44fd13e638fb admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:56.421984 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 819/1660] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:43:56 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:43:56.423250 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5567614e-8af2-4a4f-959e-d543d239f27b admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:56.443605 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5567614e-8af2-4a4f-959e-d543d239f27b admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:56.461934 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f873dfe-05e3-43d8-a659-4727dcf6285b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:56.462898 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4bd7b1-b3aa-438f-a067-44fd13e638fb admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:56.464629 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f873dfe-05e3-43d8-a659-4727dcf6285b admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:56.465620 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f873dfe-05e3-43d8-a659-4727dcf6285b admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:56.465620 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f873dfe-05e3-43d8-a659-4727dcf6285b admin admin] PATH_INFO: `/v3/projects/cb8f47d178b14878b76bb952e5b3571f/users/c2f8205b1ec64438a18b3a91e3076e3f/roles/0be2075946f54c8485f1c275f746478a` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:56.468982 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-0b4bd7b1-b3aa-438f-a067-44fd13e638fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:43:56.506218 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f873dfe-05e3-43d8-a659-4727dcf6285b admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb8f47d178b14878b76bb952e5b3571f, user_id=c2f8205b1ec64438a18b3a91e3076e3f, role_id=0be2075946f54c8485f1c275f746478a)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:56.507463 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4bd7b1-b3aa-438f-a067-44fd13e638fb admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:56.521584 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f873dfe-05e3-43d8-a659-4727dcf6285b admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:56.524820 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e5886f-38be-44b9-865d-cf1f4246fe58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:56.525837 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e5886f-38be-44b9-865d-cf1f4246fe58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:56.526216 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e5886f-38be-44b9-865d-cf1f4246fe58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:56.526599 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e5886f-38be-44b9-865d-cf1f4246fe58 admin admin] PATH_INFO: `/v3/projects/e093a761d7014c8d81a9a8e3202637a3` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:56.528224 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e5886f-38be-44b9-865d-cf1f4246fe58 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e093a761d7014c8d81a9a8e3202637a3)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:56.530311 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e5886f-38be-44b9-865d-cf1f4246fe58 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:56.553751 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 842/1661] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:43:56 2022] PUT /identity/v3/projects/cb8f47d178b14878b76bb952e5b3571f/users/c2f8205b1ec64438a18b3a91e3076e3f/roles/0be2075946f54c8485f1c275f746478a => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:43:56.566168 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5aef33-8bb9-4e9a-a82d-cca465e1bdd8 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:56.566548 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5aef33-8bb9-4e9a-a82d-cca465e1bdd8 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:56.567371 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5aef33-8bb9-4e9a-a82d-cca465e1bdd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:56.619346 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-bd5aef33-8bb9-4e9a-a82d-cca465e1bdd8 None None] MFA Rules not processed for user `c2f8205b1ec64438a18b3a91e3076e3f`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:43:56.620579 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-bd5aef33-8bb9-4e9a-a82d-cca465e1bdd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:43:56.663629 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-17e5886f-38be-44b9-865d-cf1f4246fe58 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e093a761d7014c8d81a9a8e3202637a3', 'request_id': 'req-17e5886f-38be-44b9-865d-cf1f4246fe58'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:43:56.686763 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-17e5886f-38be-44b9-865d-cf1f4246fe58 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e093a761d7014c8d81a9a8e3202637a3', 'request_id': 'req-17e5886f-38be-44b9-865d-cf1f4246fe58'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:43:56.697683 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-17e5886f-38be-44b9-865d-cf1f4246fe58 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e093a761d7014c8d81a9a8e3202637a3', 'request_id': 'req-17e5886f-38be-44b9-865d-cf1f4246fe58'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:43:56.718135 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-17e5886f-38be-44b9-865d-cf1f4246fe58 admin admin] The token cache is being invalidate because project e093a761d7014c8d81a9a8e3202637a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80357) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:43:56.718558 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-17e5886f-38be-44b9-865d-cf1f4246fe58 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:43:56.719489 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-17e5886f-38be-44b9-865d-cf1f4246fe58 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:43:56.726377 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 820/1662] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jun 16 17:43:56 2022] DELETE /identity/v3/projects/e093a761d7014c8d81a9a8e3202637a3 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:43:56.728219 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e5886f-38be-44b9-865d-cf1f4246fe58 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:56.736690 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-17e5886f-38be-44b9-865d-cf1f4246fe58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:43:56.749305 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:43:56.807108 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 843/1663] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:43:56 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:43:56.810571 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5aef33-8bb9-4e9a-a82d-cca465e1bdd8 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:56.813458 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-bd5aef33-8bb9-4e9a-a82d-cca465e1bdd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:43:56.875004 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:43:56.911695 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e5886f-38be-44b9-865d-cf1f4246fe58 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:56.927534 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eda0e48-350d-4f1f-831e-4fdace8b5a18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:56.928779 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eda0e48-350d-4f1f-831e-4fdace8b5a18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:56.929951 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eda0e48-350d-4f1f-831e-4fdace8b5a18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:56.930334 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eda0e48-350d-4f1f-831e-4fdace8b5a18 admin admin] PATH_INFO: `/v3/users/a9a1d23a442b43929b9cf03bdc0e6be6` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:56.932266 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eda0e48-350d-4f1f-831e-4fdace8b5a18 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9a1d23a442b43929b9cf03bdc0e6be6)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:56.938772 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eda0e48-350d-4f1f-831e-4fdace8b5a18 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:57.038632 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5aef33-8bb9-4e9a-a82d-cca465e1bdd8 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:57.055458 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215b279e-1bbf-40ec-a632-c9ded0124577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:57.074480 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215b279e-1bbf-40ec-a632-c9ded0124577 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:57.075179 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215b279e-1bbf-40ec-a632-c9ded0124577 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:57.075513 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215b279e-1bbf-40ec-a632-c9ded0124577 admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:57.078940 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215b279e-1bbf-40ec-a632-c9ded0124577 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:57.083447 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215b279e-1bbf-40ec-a632-c9ded0124577 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:57.096497 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-5eda0e48-350d-4f1f-831e-4fdace8b5a18 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9a1d23a442b43929b9cf03bdc0e6be6'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:43:57.099146 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-5eda0e48-350d-4f1f-831e-4fdace8b5a18 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9a1d23a442b43929b9cf03bdc0e6be6'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:43:57.125507 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-5eda0e48-350d-4f1f-831e-4fdace8b5a18 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9a1d23a442b43929b9cf03bdc0e6be6'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:43:57.126430 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-5eda0e48-350d-4f1f-831e-4fdace8b5a18 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9a1d23a442b43929b9cf03bdc0e6be6'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:43:57.163198 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-5eda0e48-350d-4f1f-831e-4fdace8b5a18 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9a1d23a442b43929b9cf03bdc0e6be6'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:43:57.166190 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 844/1664] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jun 16 17:43:56 2022] POST /identity/v3/projects => generated 375 bytes in 357 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:43:57.170767 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215b279e-1bbf-40ec-a632-c9ded0124577 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:57.186048 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-215b279e-1bbf-40ec-a632-c9ded0124577 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:43:57.226288 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 821/1665] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jun 16 17:43:56 2022] DELETE /identity/v3/users/a9a1d23a442b43929b9cf03bdc0e6be6 => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:43:57.399503 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215b279e-1bbf-40ec-a632-c9ded0124577 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:57.494563 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9f2581-6796-4fe0-aa56-aa7dd7679248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:57.498053 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9f2581-6796-4fe0-aa56-aa7dd7679248 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:57.498521 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9f2581-6796-4fe0-aa56-aa7dd7679248 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:57.498943 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9f2581-6796-4fe0-aa56-aa7dd7679248 admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:57.500325 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9f2581-6796-4fe0-aa56-aa7dd7679248 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:57.502555 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9f2581-6796-4fe0-aa56-aa7dd7679248 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:57.605686 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 845/1666] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jun 16 17:43:57 2022] POST /identity/v3/users => generated 294 bytes in 435 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:43:57.606814 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eda0e48-350d-4f1f-831e-4fdace8b5a18 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:57.629903 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eda0e48-350d-4f1f-831e-4fdace8b5a18 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:57.643532 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d527717-9c24-4430-a389-b1694bc55b08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:57.644592 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d527717-9c24-4430-a389-b1694bc55b08 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:57.649734 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d527717-9c24-4430-a389-b1694bc55b08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:57.650055 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d527717-9c24-4430-a389-b1694bc55b08 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:57.656078 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d527717-9c24-4430-a389-b1694bc55b08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:57.658429 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d527717-9c24-4430-a389-b1694bc55b08 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:57.697839 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9f2581-6796-4fe0-aa56-aa7dd7679248 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:57.703664 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 822/1667] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:43:57 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:43:57.733435 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9f2581-6796-4fe0-aa56-aa7dd7679248 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:57.766357 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981087c2-5a02-46d9-97a9-63547dd59428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:57.772167 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981087c2-5a02-46d9-97a9-63547dd59428 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:57.773362 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981087c2-5a02-46d9-97a9-63547dd59428 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:57.773746 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981087c2-5a02-46d9-97a9-63547dd59428 admin admin] PATH_INFO: `/v3/projects/7c2d0996d43f43c6a87d3f3c9e427a83/users/a3c09123a78442c587b3008078ef4f82/roles/3ca2dcbbd29d4a31a3ab78f2299bc239` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:57.835566 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981087c2-5a02-46d9-97a9-63547dd59428 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c2d0996d43f43c6a87d3f3c9e427a83, user_id=a3c09123a78442c587b3008078ef4f82, role_id=3ca2dcbbd29d4a31a3ab78f2299bc239)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:57.838457 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981087c2-5a02-46d9-97a9-63547dd59428 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:57.879402 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 846/1668] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:43:57 2022] PUT /identity/v3/projects/7c2d0996d43f43c6a87d3f3c9e427a83/users/a3c09123a78442c587b3008078ef4f82/roles/3ca2dcbbd29d4a31a3ab78f2299bc239 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:43:57.884149 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d527717-9c24-4430-a389-b1694bc55b08 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:57.904159 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d527717-9c24-4430-a389-b1694bc55b08 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:58.088537 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca91d669-ebe6-464c-b093-36c45c46ee5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:58.089235 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca91d669-ebe6-464c-b093-36c45c46ee5c admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:58.089527 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca91d669-ebe6-464c-b093-36c45c46ee5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:58.089834 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca91d669-ebe6-464c-b093-36c45c46ee5c admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:58.090579 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca91d669-ebe6-464c-b093-36c45c46ee5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:58.093485 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca91d669-ebe6-464c-b093-36c45c46ee5c admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:58.144656 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 823/1669] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:43:57 2022] GET /identity/v3/roles => generated 1420 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:43:58.147864 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981087c2-5a02-46d9-97a9-63547dd59428 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:58.169391 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981087c2-5a02-46d9-97a9-63547dd59428 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:58.183225 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2d355e-2923-4b68-8b82-80109eac2d86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:58.184513 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2d355e-2923-4b68-8b82-80109eac2d86 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:58.184901 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2d355e-2923-4b68-8b82-80109eac2d86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:58.185283 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2d355e-2923-4b68-8b82-80109eac2d86 admin admin] PATH_INFO: `/v3/projects/7c2d0996d43f43c6a87d3f3c9e427a83/users/a3c09123a78442c587b3008078ef4f82/roles/3ca2dcbbd29d4a31a3ab78f2299bc239` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:58.191116 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2d355e-2923-4b68-8b82-80109eac2d86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c2d0996d43f43c6a87d3f3c9e427a83, user_id=a3c09123a78442c587b3008078ef4f82, role_id=3ca2dcbbd29d4a31a3ab78f2299bc239)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:58.198595 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2d355e-2923-4b68-8b82-80109eac2d86 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:58.226893 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:43:58.257280 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 847/1670] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:43:58 2022] PUT /identity/v3/projects/7c2d0996d43f43c6a87d3f3c9e427a83/users/a3c09123a78442c587b3008078ef4f82/roles/3ca2dcbbd29d4a31a3ab78f2299bc239 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:43:58.263331 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56beb12-39d8-4419-8055-82975e856ad5 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:58.264242 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56beb12-39d8-4419-8055-82975e856ad5 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:58.264349 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56beb12-39d8-4419-8055-82975e856ad5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:58.301876 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:43:58.357465 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-e56beb12-39d8-4419-8055-82975e856ad5 None None] MFA Rules not processed for user `a3c09123a78442c587b3008078ef4f82`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:43:58.367290 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-e56beb12-39d8-4419-8055-82975e856ad5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:43:58.598698 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 824/1671] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:43:58 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 343 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:43:58.612851 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2d355e-2923-4b68-8b82-80109eac2d86 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:43:58.616586 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-1c2d355e-2923-4b68-8b82-80109eac2d86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:43:58.878110 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2d355e-2923-4b68-8b82-80109eac2d86 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:43:58.918809 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab361c7-a436-44a0-bb70-6dacff213fe5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24d5e2637b7043e08313909d5b06e329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:43:58.919509 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab361c7-a436-44a0-bb70-6dacff213fe5 service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:43:58.919802 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab361c7-a436-44a0-bb70-6dacff213fe5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:43:58.920462 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab361c7-a436-44a0-bb70-6dacff213fe5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:43:58.923742 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-5ab361c7-a436-44a0-bb70-6dacff213fe5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:43:59.127979 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab361c7-a436-44a0-bb70-6dacff213fe5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:43:59.129562 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab361c7-a436-44a0-bb70-6dacff213fe5 service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:43:59.152807 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 848/1672] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:43:58 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 542 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:44:00.323699 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef16093-140d-44f4-a356-b5f809b007ed None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:00.324741 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef16093-140d-44f4-a356-b5f809b007ed None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:00.325269 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef16093-140d-44f4-a356-b5f809b007ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:00.390108 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-aef16093-140d-44f4-a356-b5f809b007ed None None] MFA Rules not processed for user `813dd0cf7e1d4724b903b358aaac1ea7`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:44:00.392843 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-aef16093-140d-44f4-a356-b5f809b007ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:00.524105 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 825/1673] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:44:00 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:44:00.545284 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab361c7-a436-44a0-bb70-6dacff213fe5 service cinder] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:00.549272 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-5ab361c7-a436-44a0-bb70-6dacff213fe5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:00.718153 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab361c7-a436-44a0-bb70-6dacff213fe5 service cinder] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:00.739578 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08579ca0-1ed0-4091-97ea-0a2cb41053da service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f86c16179a70474a817be9fda33ece9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:00.741463 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08579ca0-1ed0-4091-97ea-0a2cb41053da service nova] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:00.744025 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08579ca0-1ed0-4091-97ea-0a2cb41053da service nova] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:00.744912 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08579ca0-1ed0-4091-97ea-0a2cb41053da service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:00.770195 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08579ca0-1ed0-4091-97ea-0a2cb41053da service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:00.774546 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08579ca0-1ed0-4091-97ea-0a2cb41053da service nova] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:00.823898 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 849/1674] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:44:00 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:44:01.104909 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef16093-140d-44f4-a356-b5f809b007ed None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:01.106549 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-aef16093-140d-44f4-a356-b5f809b007ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:01.314843 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef16093-140d-44f4-a356-b5f809b007ed None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:01.328850 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80017fb-cd86-495a-9b1a-bda83ed28be2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:01.329799 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80017fb-cd86-495a-9b1a-bda83ed28be2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:01.330088 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80017fb-cd86-495a-9b1a-bda83ed28be2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:01.333075 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80017fb-cd86-495a-9b1a-bda83ed28be2 admin admin] PATH_INFO: `/v3/projects/96f4e5ab7403467bbe419ea024009a58` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:01.334795 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80017fb-cd86-495a-9b1a-bda83ed28be2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=96f4e5ab7403467bbe419ea024009a58)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:01.338660 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80017fb-cd86-495a-9b1a-bda83ed28be2 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:01.418208 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:44:01.510100 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-d80017fb-cd86-495a-9b1a-bda83ed28be2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96f4e5ab7403467bbe419ea024009a58', 'request_id': 'req-d80017fb-cd86-495a-9b1a-bda83ed28be2'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:01.540494 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-d80017fb-cd86-495a-9b1a-bda83ed28be2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96f4e5ab7403467bbe419ea024009a58', 'request_id': 'req-d80017fb-cd86-495a-9b1a-bda83ed28be2'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:01.551722 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-d80017fb-cd86-495a-9b1a-bda83ed28be2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96f4e5ab7403467bbe419ea024009a58', 'request_id': 'req-d80017fb-cd86-495a-9b1a-bda83ed28be2'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:01.590782 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-d80017fb-cd86-495a-9b1a-bda83ed28be2 admin admin] The token cache is being invalidate because project 96f4e5ab7403467bbe419ea024009a58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80357) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:44:01.591202 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-d80017fb-cd86-495a-9b1a-bda83ed28be2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:01.591804 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-d80017fb-cd86-495a-9b1a-bda83ed28be2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:01.598829 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 826/1675] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jun 16 17:44:01 2022] DELETE /identity/v3/projects/96f4e5ab7403467bbe419ea024009a58 => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:44:01.603912 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63b1d13-843d-4bd1-ae07-846966dbe91d None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:01.604432 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63b1d13-843d-4bd1-ae07-846966dbe91d None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:01.604533 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63b1d13-843d-4bd1-ae07-846966dbe91d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:01.657121 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-c63b1d13-843d-4bd1-ae07-846966dbe91d None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:44:01.662127 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-c63b1d13-843d-4bd1-ae07-846966dbe91d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:01.767676 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:44:01.844172 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 850/1676] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:44:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:44:01.851937 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9426bdd0-2c3e-4670-9940-0ca4b87d6650 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:01.852517 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9426bdd0-2c3e-4670-9940-0ca4b87d6650 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:01.852935 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9426bdd0-2c3e-4670-9940-0ca4b87d6650 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:01.912446 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-9426bdd0-2c3e-4670-9940-0ca4b87d6650 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:44:01.921968 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-9426bdd0-2c3e-4670-9940-0ca4b87d6650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:01.938235 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 827/1677] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:44:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:44:01.942595 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63b1d13-843d-4bd1-ae07-846966dbe91d None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:01.957228 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63b1d13-843d-4bd1-ae07-846966dbe91d None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:01.970544 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d4cba7-460f-497e-a10c-96b1713e8260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:01.971620 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d4cba7-460f-497e-a10c-96b1713e8260 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:01.972118 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d4cba7-460f-497e-a10c-96b1713e8260 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:01.972631 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d4cba7-460f-497e-a10c-96b1713e8260 admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:01.974359 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d4cba7-460f-497e-a10c-96b1713e8260 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:44:01.974894 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d4cba7-460f-497e-a10c-96b1713e8260 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:01.979370 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d4cba7-460f-497e-a10c-96b1713e8260 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:02.024537 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 851/1678] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jun 16 17:44:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:44:02.025797 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9426bdd0-2c3e-4670-9940-0ca4b87d6650 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:02.037814 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9426bdd0-2c3e-4670-9940-0ca4b87d6650 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:02.045505 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25528360-2ef4-405b-bf58-8b629487b716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:02.046809 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25528360-2ef4-405b-bf58-8b629487b716 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:02.048428 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25528360-2ef4-405b-bf58-8b629487b716 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:02.048736 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25528360-2ef4-405b-bf58-8b629487b716 admin admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:02.049589 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25528360-2ef4-405b-bf58-8b629487b716 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:02.059278 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25528360-2ef4-405b-bf58-8b629487b716 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:02.126607 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 828/1679] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jun 16 17:44:02 2022] POST /identity/v3/projects => generated 385 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:44:02.130438 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d4cba7-460f-497e-a10c-96b1713e8260 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:02.144290 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d4cba7-460f-497e-a10c-96b1713e8260 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:02.343017 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25528360-2ef4-405b-bf58-8b629487b716 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:02.345417 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-25528360-2ef4-405b-bf58-8b629487b716 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:02.404054 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e89ac7a-207c-4cb2-8b20-4910aaaf6c91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:02.406721 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e89ac7a-207c-4cb2-8b20-4910aaaf6c91 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:02.410486 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e89ac7a-207c-4cb2-8b20-4910aaaf6c91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:02.410486 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e89ac7a-207c-4cb2-8b20-4910aaaf6c91 admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:02.410486 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e89ac7a-207c-4cb2-8b20-4910aaaf6c91 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:02.415861 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e89ac7a-207c-4cb2-8b20-4910aaaf6c91 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:02.502941 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 852/1680] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jun 16 17:44:02 2022] POST /identity/v3/users => generated 293 bytes in 373 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:44:02.507279 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e89ac7a-207c-4cb2-8b20-4910aaaf6c91 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:02.525930 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e89ac7a-207c-4cb2-8b20-4910aaaf6c91 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:02.532889 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d81a6d-0128-4e34-89d0-6a3deaeb2088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:02.533745 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d81a6d-0128-4e34-89d0-6a3deaeb2088 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:02.534200 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d81a6d-0128-4e34-89d0-6a3deaeb2088 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:02.534665 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d81a6d-0128-4e34-89d0-6a3deaeb2088 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:02.535726 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d81a6d-0128-4e34-89d0-6a3deaeb2088 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:02.539550 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d81a6d-0128-4e34-89d0-6a3deaeb2088 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:02.557493 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 853/1681] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:44:02 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:44:02.561533 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d81a6d-0128-4e34-89d0-6a3deaeb2088 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:02.581434 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25528360-2ef4-405b-bf58-8b629487b716 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:02.588436 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e7e0bf-496c-426c-8ac3-30b823042ae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:02.590540 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e7e0bf-496c-426c-8ac3-30b823042ae5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:02.591015 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e7e0bf-496c-426c-8ac3-30b823042ae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:02.591499 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e7e0bf-496c-426c-8ac3-30b823042ae5 admin admin] PATH_INFO: `/v3/users/68cad55713924deda605b0b9ace133a1` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:02.593055 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e7e0bf-496c-426c-8ac3-30b823042ae5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=68cad55713924deda605b0b9ace133a1)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:02.597249 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e7e0bf-496c-426c-8ac3-30b823042ae5 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:02.613592 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d81a6d-0128-4e34-89d0-6a3deaeb2088 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:02.631879 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1d6057-83de-4f6e-b4a2-9dcda751537a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:02.632840 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1d6057-83de-4f6e-b4a2-9dcda751537a admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:02.633389 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1d6057-83de-4f6e-b4a2-9dcda751537a admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:02.633671 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1d6057-83de-4f6e-b4a2-9dcda751537a admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:02.634463 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1d6057-83de-4f6e-b4a2-9dcda751537a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:02.640411 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1d6057-83de-4f6e-b4a2-9dcda751537a admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:02.670459 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 854/1682] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:44:02 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:44:02.672156 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1d6057-83de-4f6e-b4a2-9dcda751537a admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:02.685619 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-40e7e0bf-496c-426c-8ac3-30b823042ae5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68cad55713924deda605b0b9ace133a1'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:02.690834 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-40e7e0bf-496c-426c-8ac3-30b823042ae5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68cad55713924deda605b0b9ace133a1'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:02.705788 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1d6057-83de-4f6e-b4a2-9dcda751537a admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:02.714106 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-40e7e0bf-496c-426c-8ac3-30b823042ae5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68cad55713924deda605b0b9ace133a1'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:02.714854 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-40e7e0bf-496c-426c-8ac3-30b823042ae5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68cad55713924deda605b0b9ace133a1'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:02.722952 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3b3cd3-e1c0-4930-8af1-44574cd4140f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:02.724140 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3b3cd3-e1c0-4930-8af1-44574cd4140f admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:02.725869 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3b3cd3-e1c0-4930-8af1-44574cd4140f admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:02.726865 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3b3cd3-e1c0-4930-8af1-44574cd4140f admin admin] PATH_INFO: `/v3/projects/825a99cef64b41359074352d584c93f5/users/874534107ea34577bf278c2c73b821af/roles/0be2075946f54c8485f1c275f746478a` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:02.750869 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-40e7e0bf-496c-426c-8ac3-30b823042ae5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68cad55713924deda605b0b9ace133a1'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:02.767276 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3b3cd3-e1c0-4930-8af1-44574cd4140f admin admin] RBAC: Authorizing `identity:create_grant(project_id=825a99cef64b41359074352d584c93f5, user_id=874534107ea34577bf278c2c73b821af, role_id=0be2075946f54c8485f1c275f746478a)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:02.775925 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3b3cd3-e1c0-4930-8af1-44574cd4140f admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:02.785435 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 829/1683] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jun 16 17:44:02 2022] DELETE /identity/v3/users/68cad55713924deda605b0b9ace133a1 => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:44:02.789620 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4641663-b360-4ccb-846f-f9265d152e71 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:02.790250 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4641663-b360-4ccb-846f-f9265d152e71 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:02.791528 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4641663-b360-4ccb-846f-f9265d152e71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:02.809250 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 855/1684] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:44:02 2022] PUT /identity/v3/projects/825a99cef64b41359074352d584c93f5/users/874534107ea34577bf278c2c73b821af/roles/0be2075946f54c8485f1c275f746478a => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:44:02.822896 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09841ca9-dcbc-4edf-bd04-be6e9c85d507 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:02.823704 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09841ca9-dcbc-4edf-bd04-be6e9c85d507 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:02.823886 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=81,keystone:DELETE=16,keystone:INSERT=3 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:44:02.823992 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09841ca9-dcbc-4edf-bd04-be6e9c85d507 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:02.857933 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-f4641663-b360-4ccb-846f-f9265d152e71 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:44:02.872070 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-f4641663-b360-4ccb-846f-f9265d152e71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:02.923216 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-09841ca9-dcbc-4edf-bd04-be6e9c85d507 None None] MFA Rules not processed for user `874534107ea34577bf278c2c73b821af`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:44:02.925598 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-09841ca9-dcbc-4edf-bd04-be6e9c85d507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:03.091893 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 830/1685] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:44:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:44:03.119845 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4641663-b360-4ccb-846f-f9265d152e71 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:03.121605 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-f4641663-b360-4ccb-846f-f9265d152e71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:03.123986 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:44:03.138227 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 856/1686] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:44:02 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:44:03.144066 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09841ca9-dcbc-4edf-bd04-be6e9c85d507 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:03.146596 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-09841ca9-dcbc-4edf-bd04-be6e9c85d507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:03.194024 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09841ca9-dcbc-4edf-bd04-be6e9c85d507 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:03.203091 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c902b0-72b6-4b30-b61f-177c2fd19877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:03.204930 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c902b0-72b6-4b30-b61f-177c2fd19877 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:03.205412 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c902b0-72b6-4b30-b61f-177c2fd19877 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:03.205929 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c902b0-72b6-4b30-b61f-177c2fd19877 admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:03.208703 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c902b0-72b6-4b30-b61f-177c2fd19877 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:03.213833 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c902b0-72b6-4b30-b61f-177c2fd19877 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:03.271068 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4641663-b360-4ccb-846f-f9265d152e71 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:03.277723 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351d9cfa-6481-4754-ba11-19b5a351e6bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:03.278411 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351d9cfa-6481-4754-ba11-19b5a351e6bf service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:03.278700 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351d9cfa-6481-4754-ba11-19b5a351e6bf service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:03.278983 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351d9cfa-6481-4754-ba11-19b5a351e6bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:03.281886 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 857/1687] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jun 16 17:44:03 2022] POST /identity/v3/projects => generated 387 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:44:03.288280 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c902b0-72b6-4b30-b61f-177c2fd19877 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:03.311395 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351d9cfa-6481-4754-ba11-19b5a351e6bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:03.329474 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351d9cfa-6481-4754-ba11-19b5a351e6bf service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:03.350626 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 831/1688] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:44:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:44:03.417213 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c902b0-72b6-4b30-b61f-177c2fd19877 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:03.437684 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce72d77-e138-44f7-8129-4a44111e9039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:03.438733 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce72d77-e138-44f7-8129-4a44111e9039 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:03.439190 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce72d77-e138-44f7-8129-4a44111e9039 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:03.439777 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce72d77-e138-44f7-8129-4a44111e9039 admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:03.441598 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce72d77-e138-44f7-8129-4a44111e9039 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:03.446998 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce72d77-e138-44f7-8129-4a44111e9039 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:03.485447 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 858/1689] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jun 16 17:44:03 2022] POST /identity/v3/users => generated 300 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:44:03.486321 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351d9cfa-6481-4754-ba11-19b5a351e6bf service neutron] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:03.501781 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351d9cfa-6481-4754-ba11-19b5a351e6bf service neutron] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:03.516348 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f08dea8-9e21-41e9-8d86-c84df292937a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:03.517470 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f08dea8-9e21-41e9-8d86-c84df292937a admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:03.517470 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f08dea8-9e21-41e9-8d86-c84df292937a admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:03.517791 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f08dea8-9e21-41e9-8d86-c84df292937a admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:03.518988 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f08dea8-9e21-41e9-8d86-c84df292937a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:03.520162 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f08dea8-9e21-41e9-8d86-c84df292937a admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:03.548614 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 832/1690] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:44:03 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 17:44:03.551990 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce72d77-e138-44f7-8129-4a44111e9039 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:03.566570 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce72d77-e138-44f7-8129-4a44111e9039 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:03.587215 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac1ad9f-428f-471b-ab3e-ec5a58fb8d34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:03.588694 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac1ad9f-428f-471b-ab3e-ec5a58fb8d34 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:03.588694 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac1ad9f-428f-471b-ab3e-ec5a58fb8d34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:03.588863 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac1ad9f-428f-471b-ab3e-ec5a58fb8d34 admin admin] PATH_INFO: `/v3/projects/27c6ee68a34e4851be609a5d269f38de/users/e1dd8a16a2fd4176b2aea350ed612118/roles/3ca2dcbbd29d4a31a3ab78f2299bc239` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:03.627070 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac1ad9f-428f-471b-ab3e-ec5a58fb8d34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27c6ee68a34e4851be609a5d269f38de, user_id=e1dd8a16a2fd4176b2aea350ed612118, role_id=3ca2dcbbd29d4a31a3ab78f2299bc239)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:03.629937 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac1ad9f-428f-471b-ab3e-ec5a58fb8d34 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:03.659257 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 859/1691] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:44:03 2022] PUT /identity/v3/projects/27c6ee68a34e4851be609a5d269f38de/users/e1dd8a16a2fd4176b2aea350ed612118/roles/3ca2dcbbd29d4a31a3ab78f2299bc239 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:44:03.663028 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f08dea8-9e21-41e9-8d86-c84df292937a admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:03.676794 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f08dea8-9e21-41e9-8d86-c84df292937a admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:03.840333 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80df0ebd-db6d-4ee2-b74e-420abfaf8eaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:03.841083 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80df0ebd-db6d-4ee2-b74e-420abfaf8eaa admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:03.841368 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80df0ebd-db6d-4ee2-b74e-420abfaf8eaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:03.841644 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80df0ebd-db6d-4ee2-b74e-420abfaf8eaa admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:03.842449 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80df0ebd-db6d-4ee2-b74e-420abfaf8eaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:03.844542 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80df0ebd-db6d-4ee2-b74e-420abfaf8eaa admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:03.876189 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 833/1692] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:44:03 2022] GET /identity/v3/roles => generated 1420 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:44:03.879441 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac1ad9f-428f-471b-ab3e-ec5a58fb8d34 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:03.900286 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac1ad9f-428f-471b-ab3e-ec5a58fb8d34 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:03.913713 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df33920-d8be-4a5f-b52e-291a50cc7155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:03.915502 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df33920-d8be-4a5f-b52e-291a50cc7155 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:03.916182 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df33920-d8be-4a5f-b52e-291a50cc7155 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:03.916683 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df33920-d8be-4a5f-b52e-291a50cc7155 admin admin] PATH_INFO: `/v3/projects/27c6ee68a34e4851be609a5d269f38de/users/e1dd8a16a2fd4176b2aea350ed612118/roles/3ca2dcbbd29d4a31a3ab78f2299bc239` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:03.919539 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df33920-d8be-4a5f-b52e-291a50cc7155 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27c6ee68a34e4851be609a5d269f38de, user_id=e1dd8a16a2fd4176b2aea350ed612118, role_id=3ca2dcbbd29d4a31a3ab78f2299bc239)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:03.930315 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df33920-d8be-4a5f-b52e-291a50cc7155 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:03.973311 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 860/1693] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:44:03 2022] PUT /identity/v3/projects/27c6ee68a34e4851be609a5d269f38de/users/e1dd8a16a2fd4176b2aea350ed612118/roles/3ca2dcbbd29d4a31a3ab78f2299bc239 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:44:03.986582 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743bd7d0-e433-4c8a-85a7-0dc631254fe6 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:03.986582 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743bd7d0-e433-4c8a-85a7-0dc631254fe6 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:03.986582 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743bd7d0-e433-4c8a-85a7-0dc631254fe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:04.062885 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-743bd7d0-e433-4c8a-85a7-0dc631254fe6 None None] MFA Rules not processed for user `e1dd8a16a2fd4176b2aea350ed612118`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:44:04.063757 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-743bd7d0-e433-4c8a-85a7-0dc631254fe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:04.106938 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:44:04.247267 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 834/1694] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:44:03 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:44:04.259896 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df33920-d8be-4a5f-b52e-291a50cc7155 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:04.262060 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-9df33920-d8be-4a5f-b52e-291a50cc7155 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:04.445186 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df33920-d8be-4a5f-b52e-291a50cc7155 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:04.455783 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecb3c05-e7db-419c-a607-b8627942a70d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24d5e2637b7043e08313909d5b06e329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:04.456487 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecb3c05-e7db-419c-a607-b8627942a70d service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:04.456772 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecb3c05-e7db-419c-a607-b8627942a70d service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:04.457047 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecb3c05-e7db-419c-a607-b8627942a70d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:04.505004 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecb3c05-e7db-419c-a607-b8627942a70d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:04.506803 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecb3c05-e7db-419c-a607-b8627942a70d service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:04.524082 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 861/1695] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:44:04 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:44:04.575184 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743bd7d0-e433-4c8a-85a7-0dc631254fe6 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:04.741956 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743bd7d0-e433-4c8a-85a7-0dc631254fe6 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:04.755402 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bd8b82-61c5-49ae-8d07-132c376c0e4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:04.756626 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bd8b82-61c5-49ae-8d07-132c376c0e4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:04.758319 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bd8b82-61c5-49ae-8d07-132c376c0e4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:04.758319 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bd8b82-61c5-49ae-8d07-132c376c0e4b admin admin] PATH_INFO: `/v3/users/874534107ea34577bf278c2c73b821af` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:04.760529 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bd8b82-61c5-49ae-8d07-132c376c0e4b admin admin] RBAC: Authorizing `identity:delete_user(user_id=874534107ea34577bf278c2c73b821af)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:04.772927 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bd8b82-61c5-49ae-8d07-132c376c0e4b admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:04.889170 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae120092-080f-4221-bbb5-9990c2396dc2 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:04.889678 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae120092-080f-4221-bbb5-9990c2396dc2 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:04.890069 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae120092-080f-4221-bbb5-9990c2396dc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:04.903095 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-d1bd8b82-61c5-49ae-8d07-132c376c0e4b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '874534107ea34577bf278c2c73b821af'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:04.905079 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-d1bd8b82-61c5-49ae-8d07-132c376c0e4b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '874534107ea34577bf278c2c73b821af'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:04.924891 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-d1bd8b82-61c5-49ae-8d07-132c376c0e4b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '874534107ea34577bf278c2c73b821af'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:04.926375 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-d1bd8b82-61c5-49ae-8d07-132c376c0e4b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '874534107ea34577bf278c2c73b821af'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:04.978201 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-d1bd8b82-61c5-49ae-8d07-132c376c0e4b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '874534107ea34577bf278c2c73b821af'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:04.978413 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-ae120092-080f-4221-bbb5-9990c2396dc2 None None] MFA Rules not processed for user `e577b2f2f18a4be397b6ea6e7f1ba0ad`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:44:04.981796 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-ae120092-080f-4221-bbb5-9990c2396dc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:05.038265 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 835/1696] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jun 16 17:44:04 2022] DELETE /identity/v3/users/874534107ea34577bf278c2c73b821af => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:44:05.045594 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5f6b7f-1c20-490d-abb5-19808468fd9d None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:05.046173 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5f6b7f-1c20-490d-abb5-19808468fd9d None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:05.046610 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5f6b7f-1c20-490d-abb5-19808468fd9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:05.047092 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:44:05.070514 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-bd5f6b7f-1c20-490d-abb5-19808468fd9d None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:44:05.071008 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-bd5f6b7f-1c20-490d-abb5-19808468fd9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:05.138366 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:44:05.214366 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 836/1697] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:44:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:44:05.223611 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 862/1698] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:44:04 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 343 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:44:05.239862 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5f6b7f-1c20-490d-abb5-19808468fd9d None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:05.240050 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d510d2-9ea4-4a70-ad94-2cdb8589ef81 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:05.240050 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d510d2-9ea4-4a70-ad94-2cdb8589ef81 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:05.240582 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d510d2-9ea4-4a70-ad94-2cdb8589ef81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:05.241351 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-bd5f6b7f-1c20-490d-abb5-19808468fd9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:05.288993 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-d5d510d2-9ea4-4a70-ad94-2cdb8589ef81 None None] MFA Rules not processed for user `e577b2f2f18a4be397b6ea6e7f1ba0ad`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:44:05.295223 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-d5d510d2-9ea4-4a70-ad94-2cdb8589ef81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:05.449022 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5f6b7f-1c20-490d-abb5-19808468fd9d None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:05.456902 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 863/1699] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:44:05 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:44:05.457333 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294ccc1f-f2a0-4f5b-b777-f2868e4079f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:05.458088 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294ccc1f-f2a0-4f5b-b777-f2868e4079f5 service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:05.458811 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294ccc1f-f2a0-4f5b-b777-f2868e4079f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:05.458811 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294ccc1f-f2a0-4f5b-b777-f2868e4079f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:05.482438 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d510d2-9ea4-4a70-ad94-2cdb8589ef81 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:05.485564 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-d5d510d2-9ea4-4a70-ad94-2cdb8589ef81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:05.497431 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294ccc1f-f2a0-4f5b-b777-f2868e4079f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:05.501883 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294ccc1f-f2a0-4f5b-b777-f2868e4079f5 service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:05.525417 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 837/1700] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:44:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:44:05.598812 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d510d2-9ea4-4a70-ad94-2cdb8589ef81 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:05.607358 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5d030a-1d8b-419d-b959-41371ce4c4aa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f86c16179a70474a817be9fda33ece9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:05.609158 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5d030a-1d8b-419d-b959-41371ce4c4aa service nova] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:05.609158 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5d030a-1d8b-419d-b959-41371ce4c4aa service nova] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:05.609366 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5d030a-1d8b-419d-b959-41371ce4c4aa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:05.634510 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5d030a-1d8b-419d-b959-41371ce4c4aa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:05.638970 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5d030a-1d8b-419d-b959-41371ce4c4aa service nova] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:05.664137 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038ea2f1-99d2-4af1-8c7a-5674e43fb614 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:05.665155 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038ea2f1-99d2-4af1-8c7a-5674e43fb614 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:05.665610 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038ea2f1-99d2-4af1-8c7a-5674e43fb614 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:05.702512 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 864/1701] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:44:05 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 17:44:05.705424 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5d030a-1d8b-419d-b959-41371ce4c4aa service nova] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:05.712564 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-db5d030a-1d8b-419d-b959-41371ce4c4aa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:05.754084 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-038ea2f1-99d2-4af1-8c7a-5674e43fb614 None None] MFA Rules not processed for user `8fff53cb0d084d9284825488f6843ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:44:05.755316 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-038ea2f1-99d2-4af1-8c7a-5674e43fb614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:05.767614 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5d030a-1d8b-419d-b959-41371ce4c4aa service nova] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:05.786610 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f4c18f-c5bb-47df-8376-6dda1eb941a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:05.788878 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f4c18f-c5bb-47df-8376-6dda1eb941a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:05.792025 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f4c18f-c5bb-47df-8376-6dda1eb941a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:05.794347 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f4c18f-c5bb-47df-8376-6dda1eb941a6 admin admin] PATH_INFO: `/v3/projects/9a7c548d6ba8472fb04ed3999ef86afc` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:05.799721 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f4c18f-c5bb-47df-8376-6dda1eb941a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a7c548d6ba8472fb04ed3999ef86afc)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:05.809055 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f4c18f-c5bb-47df-8376-6dda1eb941a6 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:05.962565 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:44:05.965824 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 838/1702] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:44:05 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:44:05.974397 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038ea2f1-99d2-4af1-8c7a-5674e43fb614 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:05.976287 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-038ea2f1-99d2-4af1-8c7a-5674e43fb614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:05.979842 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-b0f4c18f-c5bb-47df-8376-6dda1eb941a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a7c548d6ba8472fb04ed3999ef86afc', 'request_id': 'req-b0f4c18f-c5bb-47df-8376-6dda1eb941a6'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:06.002232 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-b0f4c18f-c5bb-47df-8376-6dda1eb941a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a7c548d6ba8472fb04ed3999ef86afc', 'request_id': 'req-b0f4c18f-c5bb-47df-8376-6dda1eb941a6'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:06.025034 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-b0f4c18f-c5bb-47df-8376-6dda1eb941a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a7c548d6ba8472fb04ed3999ef86afc', 'request_id': 'req-b0f4c18f-c5bb-47df-8376-6dda1eb941a6'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:06.032262 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038ea2f1-99d2-4af1-8c7a-5674e43fb614 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:06.074372 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-b0f4c18f-c5bb-47df-8376-6dda1eb941a6 admin admin] The token cache is being invalidate because project 9a7c548d6ba8472fb04ed3999ef86afc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:44:06.074676 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-b0f4c18f-c5bb-47df-8376-6dda1eb941a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:06.076786 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-b0f4c18f-c5bb-47df-8376-6dda1eb941a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:06.082800 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 865/1703] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jun 16 17:44:05 2022] DELETE /identity/v3/projects/9a7c548d6ba8472fb04ed3999ef86afc => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:44:06.087509 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685894a3-9db1-40c9-a0d1-a3f7b3638e33 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:06.087803 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685894a3-9db1-40c9-a0d1-a3f7b3638e33 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:06.088097 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685894a3-9db1-40c9-a0d1-a3f7b3638e33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:06.174266 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-685894a3-9db1-40c9-a0d1-a3f7b3638e33 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:44:06.175871 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-685894a3-9db1-40c9-a0d1-a3f7b3638e33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:06.186765 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34be925c-fbe5-4bae-a867-0e40948fc4a7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f86c16179a70474a817be9fda33ece9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:06.191246 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34be925c-fbe5-4bae-a867-0e40948fc4a7 service nova] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:06.191246 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34be925c-fbe5-4bae-a867-0e40948fc4a7 service nova] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:06.191526 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34be925c-fbe5-4bae-a867-0e40948fc4a7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:06.198389 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-34be925c-fbe5-4bae-a867-0e40948fc4a7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:06.216104 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:44:06.355996 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 866/1704] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:44:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:44:06.358043 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685894a3-9db1-40c9-a0d1-a3f7b3638e33 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:06.370741 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-685894a3-9db1-40c9-a0d1-a3f7b3638e33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:06.391841 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34be925c-fbe5-4bae-a867-0e40948fc4a7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:06.396720 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34be925c-fbe5-4bae-a867-0e40948fc4a7 service nova] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:06.442788 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 839/1705] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:44:05 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 17:44:06.445809 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc0e530-f7e4-46f3-a138-cfe5f8fdc2d1 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:06.453008 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc0e530-f7e4-46f3-a138-cfe5f8fdc2d1 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:06.453008 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc0e530-f7e4-46f3-a138-cfe5f8fdc2d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:06.532969 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-3dc0e530-f7e4-46f3-a138-cfe5f8fdc2d1 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:44:06.534613 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-3dc0e530-f7e4-46f3-a138-cfe5f8fdc2d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:06.557474 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 840/1706] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:44:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:44:06.563281 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc0e530-f7e4-46f3-a138-cfe5f8fdc2d1 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:06.595310 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc0e530-f7e4-46f3-a138-cfe5f8fdc2d1 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:06.617777 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685894a3-9db1-40c9-a0d1-a3f7b3638e33 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:06.622510 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700a495b-7b4f-4ec8-8ca9-d4ccabbca3fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:06.625362 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617cfc46-921f-4bcf-a18c-5eef26868183 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24d5e2637b7043e08313909d5b06e329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:06.626747 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617cfc46-921f-4bcf-a18c-5eef26868183 service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:06.627467 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617cfc46-921f-4bcf-a18c-5eef26868183 service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:06.627467 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617cfc46-921f-4bcf-a18c-5eef26868183 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:06.628353 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700a495b-7b4f-4ec8-8ca9-d4ccabbca3fd admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:06.629465 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-617cfc46-921f-4bcf-a18c-5eef26868183 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:06.629971 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700a495b-7b4f-4ec8-8ca9-d4ccabbca3fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:06.633924 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700a495b-7b4f-4ec8-8ca9-d4ccabbca3fd admin admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:06.636991 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700a495b-7b4f-4ec8-8ca9-d4ccabbca3fd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:44:06.637560 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700a495b-7b4f-4ec8-8ca9-d4ccabbca3fd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:06.642769 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700a495b-7b4f-4ec8-8ca9-d4ccabbca3fd admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:06.692229 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 841/1707] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jun 16 17:44:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:44:06.697516 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700a495b-7b4f-4ec8-8ca9-d4ccabbca3fd admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:06.738484 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700a495b-7b4f-4ec8-8ca9-d4ccabbca3fd admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:06.763347 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6a4cde-23e6-4cba-8796-610626963e6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:06.764070 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6a4cde-23e6-4cba-8796-610626963e6a admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:06.764352 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6a4cde-23e6-4cba-8796-610626963e6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:06.766223 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6a4cde-23e6-4cba-8796-610626963e6a admin admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:06.766940 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6a4cde-23e6-4cba-8796-610626963e6a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:06.768921 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6a4cde-23e6-4cba-8796-610626963e6a admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:06.771695 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617cfc46-921f-4bcf-a18c-5eef26868183 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:06.773192 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617cfc46-921f-4bcf-a18c-5eef26868183 service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:06.812752 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 867/1708] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:44:06 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:44:06.850294 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 842/1709] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jun 16 17:44:06 2022] POST /identity/v3/projects => generated 383 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:44:06.857048 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617cfc46-921f-4bcf-a18c-5eef26868183 service cinder] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:06.888760 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617cfc46-921f-4bcf-a18c-5eef26868183 service cinder] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:06.945094 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:44:07.084945 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62daab5f-6654-4176-b560-b1c12d2efaa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:07.085716 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62daab5f-6654-4176-b560-b1c12d2efaa4 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:07.086067 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62daab5f-6654-4176-b560-b1c12d2efaa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:07.086668 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62daab5f-6654-4176-b560-b1c12d2efaa4 admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:07.087173 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62daab5f-6654-4176-b560-b1c12d2efaa4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:07.089270 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62daab5f-6654-4176-b560-b1c12d2efaa4 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:07.146773 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 868/1710] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jun 16 17:44:06 2022] POST /identity/v3/users => generated 292 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:44:07.151249 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6a4cde-23e6-4cba-8796-610626963e6a admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:07.170448 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6a4cde-23e6-4cba-8796-610626963e6a admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:07.199104 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c84426-9852-4e52-a719-2670bc75f346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:07.202754 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c84426-9852-4e52-a719-2670bc75f346 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:07.203093 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c84426-9852-4e52-a719-2670bc75f346 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:07.203374 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c84426-9852-4e52-a719-2670bc75f346 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:07.205465 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c84426-9852-4e52-a719-2670bc75f346 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:07.209632 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c84426-9852-4e52-a719-2670bc75f346 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:07.266749 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 843/1711] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:44:07 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:44:07.270549 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62daab5f-6654-4176-b560-b1c12d2efaa4 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:07.291555 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62daab5f-6654-4176-b560-b1c12d2efaa4 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:07.306928 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c838e975-ecef-4045-94cf-989389261bc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:07.310892 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c838e975-ecef-4045-94cf-989389261bc9 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:07.327326 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c838e975-ecef-4045-94cf-989389261bc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:07.328515 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c838e975-ecef-4045-94cf-989389261bc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:07.330663 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c838e975-ecef-4045-94cf-989389261bc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:07.336768 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c838e975-ecef-4045-94cf-989389261bc9 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:07.390063 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 869/1712] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:44:07 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:44:07.392008 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c84426-9852-4e52-a719-2670bc75f346 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:07.400394 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c838e975-ecef-4045-94cf-989389261bc9 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:07.409186 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-c838e975-ecef-4045-94cf-989389261bc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:07.417845 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c84426-9852-4e52-a719-2670bc75f346 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:07.447970 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1598132-3bb5-4e6f-9112-ed91964b9e46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:07.448938 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1598132-3bb5-4e6f-9112-ed91964b9e46 admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:07.449854 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1598132-3bb5-4e6f-9112-ed91964b9e46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:07.454577 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1598132-3bb5-4e6f-9112-ed91964b9e46 admin admin] PATH_INFO: `/v3/projects/75dc17395d7644708d5d6e808b30830c/users/2097f78edf1446108d0b7ba2943048e3/roles/0be2075946f54c8485f1c275f746478a` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:07.482085 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c838e975-ecef-4045-94cf-989389261bc9 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:07.506146 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1598132-3bb5-4e6f-9112-ed91964b9e46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75dc17395d7644708d5d6e808b30830c, user_id=2097f78edf1446108d0b7ba2943048e3, role_id=0be2075946f54c8485f1c275f746478a)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:07.509935 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4889843a-ab23-4bb4-8cd6-ebda5bb5ce59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:07.512123 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4889843a-ab23-4bb4-8cd6-ebda5bb5ce59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:07.513079 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4889843a-ab23-4bb4-8cd6-ebda5bb5ce59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:07.514486 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4889843a-ab23-4bb4-8cd6-ebda5bb5ce59 admin admin] PATH_INFO: `/v3/projects/825a99cef64b41359074352d584c93f5` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:07.517524 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1598132-3bb5-4e6f-9112-ed91964b9e46 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:07.519258 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4889843a-ab23-4bb4-8cd6-ebda5bb5ce59 admin admin] RBAC: Authorizing `identity:delete_project(project_id=825a99cef64b41359074352d584c93f5)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:07.521959 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4889843a-ab23-4bb4-8cd6-ebda5bb5ce59 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:07.554682 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 844/1713] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:44:07 2022] PUT /identity/v3/projects/75dc17395d7644708d5d6e808b30830c/users/2097f78edf1446108d0b7ba2943048e3/roles/0be2075946f54c8485f1c275f746478a => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:44:07.566439 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c328de2-b46d-4900-b47d-b4747e9fe169 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:07.567496 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c328de2-b46d-4900-b47d-b4747e9fe169 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:07.567496 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c328de2-b46d-4900-b47d-b4747e9fe169 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:07.679367 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-4889843a-ab23-4bb4-8cd6-ebda5bb5ce59 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '825a99cef64b41359074352d584c93f5', 'request_id': 'req-4889843a-ab23-4bb4-8cd6-ebda5bb5ce59'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:07.697142 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-4889843a-ab23-4bb4-8cd6-ebda5bb5ce59 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '825a99cef64b41359074352d584c93f5', 'request_id': 'req-4889843a-ab23-4bb4-8cd6-ebda5bb5ce59'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:07.707071 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-4c328de2-b46d-4900-b47d-b4747e9fe169 None None] MFA Rules not processed for user `2097f78edf1446108d0b7ba2943048e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:44:07.710994 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-4889843a-ab23-4bb4-8cd6-ebda5bb5ce59 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '825a99cef64b41359074352d584c93f5', 'request_id': 'req-4889843a-ab23-4bb4-8cd6-ebda5bb5ce59'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:07.711951 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-4c328de2-b46d-4900-b47d-b4747e9fe169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:07.751284 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-4889843a-ab23-4bb4-8cd6-ebda5bb5ce59 admin admin] The token cache is being invalidate because project 825a99cef64b41359074352d584c93f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:44:07.751791 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-4889843a-ab23-4bb4-8cd6-ebda5bb5ce59 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:07.755062 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-4889843a-ab23-4bb4-8cd6-ebda5bb5ce59 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:07.771917 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 870/1714] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jun 16 17:44:07 2022] DELETE /identity/v3/projects/825a99cef64b41359074352d584c93f5 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:44:07.771917 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4889843a-ab23-4bb4-8cd6-ebda5bb5ce59 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:07.776462 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-4889843a-ab23-4bb4-8cd6-ebda5bb5ce59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:08.006210 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 845/1715] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:44:07 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 441 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:44:08.008790 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c328de2-b46d-4900-b47d-b4747e9fe169 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:08.012304 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-4c328de2-b46d-4900-b47d-b4747e9fe169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:08.094872 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4889843a-ab23-4bb4-8cd6-ebda5bb5ce59 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:08.106756 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d10b3f2-6371-49f0-8d4d-b8033c6c124a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:08.110336 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d10b3f2-6371-49f0-8d4d-b8033c6c124a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:08.110623 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d10b3f2-6371-49f0-8d4d-b8033c6c124a admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:08.110905 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d10b3f2-6371-49f0-8d4d-b8033c6c124a admin admin] PATH_INFO: `/v3/users/e1dd8a16a2fd4176b2aea350ed612118` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:08.127064 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:44:08.128216 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d10b3f2-6371-49f0-8d4d-b8033c6c124a admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1dd8a16a2fd4176b2aea350ed612118)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:08.135807 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d10b3f2-6371-49f0-8d4d-b8033c6c124a admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:08.220250 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=84,keystone:INSERT=4,keystone:DELETE=12 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:44:08.259438 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-4d10b3f2-6371-49f0-8d4d-b8033c6c124a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1dd8a16a2fd4176b2aea350ed612118'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:08.263547 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c328de2-b46d-4900-b47d-b4747e9fe169 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:08.278282 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb96af8f-d50b-4a10-941d-06cb7cc35f52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:08.282184 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb96af8f-d50b-4a10-941d-06cb7cc35f52 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:08.282469 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb96af8f-d50b-4a10-941d-06cb7cc35f52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:08.282755 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb96af8f-d50b-4a10-941d-06cb7cc35f52 admin admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:08.283477 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb96af8f-d50b-4a10-941d-06cb7cc35f52 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:08.286839 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb96af8f-d50b-4a10-941d-06cb7cc35f52 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:08.294211 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-4d10b3f2-6371-49f0-8d4d-b8033c6c124a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1dd8a16a2fd4176b2aea350ed612118'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:08.295167 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-4d10b3f2-6371-49f0-8d4d-b8033c6c124a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1dd8a16a2fd4176b2aea350ed612118'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:08.319539 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-4d10b3f2-6371-49f0-8d4d-b8033c6c124a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1dd8a16a2fd4176b2aea350ed612118'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:08.365547 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-4d10b3f2-6371-49f0-8d4d-b8033c6c124a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1dd8a16a2fd4176b2aea350ed612118'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:08.396441 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 871/1716] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jun 16 17:44:07 2022] DELETE /identity/v3/users/e1dd8a16a2fd4176b2aea350ed612118 => generated 0 bytes in 625 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:44:08.399893 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 846/1717] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jun 16 17:44:08 2022] POST /identity/v3/projects => generated 383 bytes in 393 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:44:08.405342 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d10b3f2-6371-49f0-8d4d-b8033c6c124a admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:08.410022 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-4d10b3f2-6371-49f0-8d4d-b8033c6c124a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:08.529317 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd814143-bc3f-4661-a9bc-72180661e02e None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:08.529617 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd814143-bc3f-4661-a9bc-72180661e02e None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:08.529906 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd814143-bc3f-4661-a9bc-72180661e02e None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:08.532079 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 847/1718] 10.222.0.42 () {54 vars in 912 bytes} [Thu Jun 16 17:44:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:44:08.541802 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12423653-54c6-4572-b9c5-ec6581a3c5df None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:08.542243 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12423653-54c6-4572-b9c5-ec6581a3c5df None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:08.542536 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12423653-54c6-4572-b9c5-ec6581a3c5df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:08.638956 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d10b3f2-6371-49f0-8d4d-b8033c6c124a admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:08.671850 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-12423653-54c6-4572-b9c5-ec6581a3c5df None None] MFA Rules not processed for user `f94416103f6245888214a997b014f47a`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:44:08.675180 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-12423653-54c6-4572-b9c5-ec6581a3c5df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:08.812729 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0fd9f6-b2be-448d-9e92-03d93954462a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:08.813431 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0fd9f6-b2be-448d-9e92-03d93954462a admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:08.813721 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0fd9f6-b2be-448d-9e92-03d93954462a admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:08.814003 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0fd9f6-b2be-448d-9e92-03d93954462a admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:08.816887 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0fd9f6-b2be-448d-9e92-03d93954462a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:08.820267 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0fd9f6-b2be-448d-9e92-03d93954462a admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:08.832147 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 848/1719] 10.222.0.42 () {58 vars in 1009 bytes} [Thu Jun 16 17:44:08 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:44:08.863307 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 872/1720] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jun 16 17:44:08 2022] POST /identity/v3/users => generated 298 bytes in 458 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:44:08.866984 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12423653-54c6-4572-b9c5-ec6581a3c5df None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:08.896154 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12423653-54c6-4572-b9c5-ec6581a3c5df None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:08.909441 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0fd9f6-b2be-448d-9e92-03d93954462a admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:08.912817 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-ba0fd9f6-b2be-448d-9e92-03d93954462a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:08.946126 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee889687-8e0f-4446-97bc-eea77b7a05ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:08.946918 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee889687-8e0f-4446-97bc-eea77b7a05ad admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:08.947211 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee889687-8e0f-4446-97bc-eea77b7a05ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:08.947498 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee889687-8e0f-4446-97bc-eea77b7a05ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:08.948333 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee889687-8e0f-4446-97bc-eea77b7a05ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:08.978838 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee889687-8e0f-4446-97bc-eea77b7a05ad admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:08.985048 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0fd9f6-b2be-448d-9e92-03d93954462a admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:09.009463 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 849/1721] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:44:08 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:44:09.012915 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a21dc99-3060-4f45-98b1-d5eb3fdcd6c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:09.012915 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a21dc99-3060-4f45-98b1-d5eb3fdcd6c7 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:09.012915 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a21dc99-3060-4f45-98b1-d5eb3fdcd6c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:09.012915 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a21dc99-3060-4f45-98b1-d5eb3fdcd6c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:09.017221 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b34334-437a-4aae-b2fd-2fe7fc1ae073 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:09.017692 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b34334-437a-4aae-b2fd-2fe7fc1ae073 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:09.018106 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b34334-437a-4aae-b2fd-2fe7fc1ae073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:09.031291 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a21dc99-3060-4f45-98b1-d5eb3fdcd6c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:09.032803 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a21dc99-3060-4f45-98b1-d5eb3fdcd6c7 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:09.053889 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 873/1722] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:44:08 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:44:09.055330 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370ce691-3f0b-4131-8f0d-99c9be7f6d8c None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:09.055626 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370ce691-3f0b-4131-8f0d-99c9be7f6d8c None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:09.055972 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370ce691-3f0b-4131-8f0d-99c9be7f6d8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:09.089268 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-b4b34334-437a-4aae-b2fd-2fe7fc1ae073 None None] MFA Rules not processed for user `7b078c6bfca04de790e7b2e45739033e`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:44:09.091368 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-b4b34334-437a-4aae-b2fd-2fe7fc1ae073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:09.127586 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-370ce691-3f0b-4131-8f0d-99c9be7f6d8c None None] MFA Rules not processed for user `268c5947a265492bb7a8f181f9f5c504`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:44:09.128861 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-370ce691-3f0b-4131-8f0d-99c9be7f6d8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:09.169799 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:44:09.249649 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 850/1723] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:44:09 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:44:09.250985 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6528a364-7e9a-4267-ba87-5337e7e2911f None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:09.251274 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6528a364-7e9a-4267-ba87-5337e7e2911f None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:09.251883 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6528a364-7e9a-4267-ba87-5337e7e2911f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:09.297418 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 874/1724] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:44:09 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:44:09.299533 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-6528a364-7e9a-4267-ba87-5337e7e2911f None None] MFA Rules not processed for user `813dd0cf7e1d4724b903b358aaac1ea7`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:44:09.305927 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-6528a364-7e9a-4267-ba87-5337e7e2911f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:09.311073 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370ce691-3f0b-4131-8f0d-99c9be7f6d8c None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:09.339664 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370ce691-3f0b-4131-8f0d-99c9be7f6d8c None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:09.344847 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:44:09.346943 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9b63b1-c227-4c89-a24a-9f01d56b29f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:09.347881 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9b63b1-c227-4c89-a24a-9f01d56b29f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:09.348169 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9b63b1-c227-4c89-a24a-9f01d56b29f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:09.348494 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9b63b1-c227-4c89-a24a-9f01d56b29f0 admin admin] PATH_INFO: `/v3/projects/319de500a1894a418ebab7257eeb66c1/users/bdc09b9e889d483680a4f44f166af074/roles/3ca2dcbbd29d4a31a3ab78f2299bc239` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:09.390855 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9b63b1-c227-4c89-a24a-9f01d56b29f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=319de500a1894a418ebab7257eeb66c1, user_id=bdc09b9e889d483680a4f44f166af074, role_id=3ca2dcbbd29d4a31a3ab78f2299bc239)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:09.394025 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9b63b1-c227-4c89-a24a-9f01d56b29f0 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:09.430869 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 875/1725] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:44:09 2022] PUT /identity/v3/projects/319de500a1894a418ebab7257eeb66c1/users/bdc09b9e889d483680a4f44f166af074/roles/3ca2dcbbd29d4a31a3ab78f2299bc239 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:44:09.431840 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9b63b1-c227-4c89-a24a-9f01d56b29f0 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:09.440142 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-fe9b63b1-c227-4c89-a24a-9f01d56b29f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:09.508249 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 851/1726] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:44:09 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:44:09.510280 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1da97d-5cd6-4cef-b064-d20cda6d559e None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:09.511773 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1da97d-5cd6-4cef-b064-d20cda6d559e None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:09.513143 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1da97d-5cd6-4cef-b064-d20cda6d559e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:09.553317 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-ce1da97d-5cd6-4cef-b064-d20cda6d559e None None] MFA Rules not processed for user `268c5947a265492bb7a8f181f9f5c504`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:44:09.560515 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-ce1da97d-5cd6-4cef-b064-d20cda6d559e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:09.565844 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9b63b1-c227-4c89-a24a-9f01d56b29f0 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:09.579998 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0681a2e6-2b40-4ea8-ae54-8338d754bc73 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f86c16179a70474a817be9fda33ece9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:09.582202 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0681a2e6-2b40-4ea8-ae54-8338d754bc73 service nova] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:09.582499 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0681a2e6-2b40-4ea8-ae54-8338d754bc73 service nova] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:09.582779 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0681a2e6-2b40-4ea8-ae54-8338d754bc73 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:09.653087 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0681a2e6-2b40-4ea8-ae54-8338d754bc73 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:09.654698 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0681a2e6-2b40-4ea8-ae54-8338d754bc73 service nova] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:09.665973 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 876/1727] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:44:09 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:44:09.678323 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0681a2e6-2b40-4ea8-ae54-8338d754bc73 service nova] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:09.728509 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 852/1728] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:44:09 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:44:09.754421 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1da97d-5cd6-4cef-b064-d20cda6d559e None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:09.757185 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-ce1da97d-5cd6-4cef-b064-d20cda6d559e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:09.786489 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:44:09.859773 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0681a2e6-2b40-4ea8-ae54-8338d754bc73 service nova] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:09.872268 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35006943-fd4e-48a9-83c8-86752d0d0473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:09.873318 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35006943-fd4e-48a9-83c8-86752d0d0473 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:09.875165 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35006943-fd4e-48a9-83c8-86752d0d0473 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:09.875640 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35006943-fd4e-48a9-83c8-86752d0d0473 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:09.878011 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35006943-fd4e-48a9-83c8-86752d0d0473 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:09.881210 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35006943-fd4e-48a9-83c8-86752d0d0473 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:09.909529 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 877/1729] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:44:09 2022] GET /identity/v3/roles => generated 1420 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:44:09.915983 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35006943-fd4e-48a9-83c8-86752d0d0473 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:09.930979 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35006943-fd4e-48a9-83c8-86752d0d0473 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:09.936052 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b2b1be-c18e-4b2a-bdc2-f948a438f752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:09.937365 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b2b1be-c18e-4b2a-bdc2-f948a438f752 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:09.937758 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b2b1be-c18e-4b2a-bdc2-f948a438f752 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:09.938143 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b2b1be-c18e-4b2a-bdc2-f948a438f752 admin admin] PATH_INFO: `/v3/projects/319de500a1894a418ebab7257eeb66c1/users/bdc09b9e889d483680a4f44f166af074/roles/3ca2dcbbd29d4a31a3ab78f2299bc239` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:09.939427 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1da97d-5cd6-4cef-b064-d20cda6d559e None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:09.940792 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b2b1be-c18e-4b2a-bdc2-f948a438f752 admin admin] RBAC: Authorizing `identity:create_grant(project_id=319de500a1894a418ebab7257eeb66c1, user_id=bdc09b9e889d483680a4f44f166af074, role_id=3ca2dcbbd29d4a31a3ab78f2299bc239)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:09.943574 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b2b1be-c18e-4b2a-bdc2-f948a438f752 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:09.957908 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398759ca-5392-45a0-90e0-f80631a81fa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:09.958501 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 878/1730] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:44:09 2022] PUT /identity/v3/projects/319de500a1894a418ebab7257eeb66c1/users/bdc09b9e889d483680a4f44f166af074/roles/3ca2dcbbd29d4a31a3ab78f2299bc239 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:44:09.959033 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398759ca-5392-45a0-90e0-f80631a81fa9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:09.959215 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398759ca-5392-45a0-90e0-f80631a81fa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:09.959580 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398759ca-5392-45a0-90e0-f80631a81fa9 admin admin] PATH_INFO: `/v3/projects/27c6ee68a34e4851be609a5d269f38de` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:09.968820 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dfea77-7e1b-4b82-980d-742298b8cfe7 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:09.969296 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dfea77-7e1b-4b82-980d-742298b8cfe7 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:09.969656 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dfea77-7e1b-4b82-980d-742298b8cfe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:09.970569 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398759ca-5392-45a0-90e0-f80631a81fa9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=27c6ee68a34e4851be609a5d269f38de)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:09.973763 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398759ca-5392-45a0-90e0-f80631a81fa9 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:10.047847 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-f0dfea77-7e1b-4b82-980d-742298b8cfe7 None None] MFA Rules not processed for user `bdc09b9e889d483680a4f44f166af074`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:44:10.051954 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-f0dfea77-7e1b-4b82-980d-742298b8cfe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:10.184973 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 879/1731] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:44:09 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:44:10.190762 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-398759ca-5392-45a0-90e0-f80631a81fa9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27c6ee68a34e4851be609a5d269f38de', 'request_id': 'req-398759ca-5392-45a0-90e0-f80631a81fa9'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:10.197439 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dfea77-7e1b-4b82-980d-742298b8cfe7 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:10.200113 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-f0dfea77-7e1b-4b82-980d-742298b8cfe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:10.220650 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-398759ca-5392-45a0-90e0-f80631a81fa9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27c6ee68a34e4851be609a5d269f38de', 'request_id': 'req-398759ca-5392-45a0-90e0-f80631a81fa9'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:10.237320 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-398759ca-5392-45a0-90e0-f80631a81fa9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27c6ee68a34e4851be609a5d269f38de', 'request_id': 'req-398759ca-5392-45a0-90e0-f80631a81fa9'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:10.259253 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:44:10.290861 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-398759ca-5392-45a0-90e0-f80631a81fa9 admin admin] The token cache is being invalidate because project 27c6ee68a34e4851be609a5d269f38de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80357) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:44:10.291215 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-398759ca-5392-45a0-90e0-f80631a81fa9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:10.292348 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-398759ca-5392-45a0-90e0-f80631a81fa9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:10.294319 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 853/1732] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jun 16 17:44:09 2022] DELETE /identity/v3/projects/27c6ee68a34e4851be609a5d269f38de => generated 0 bytes in 541 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:44:10.298860 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c1612d-e9f3-4668-a129-ef4f8eee9ec6 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:10.302223 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c1612d-e9f3-4668-a129-ef4f8eee9ec6 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:10.302546 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c1612d-e9f3-4668-a129-ef4f8eee9ec6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:10.358280 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-a1c1612d-e9f3-4668-a129-ef4f8eee9ec6 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:44:10.360805 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-a1c1612d-e9f3-4668-a129-ef4f8eee9ec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:10.391142 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dfea77-7e1b-4b82-980d-742298b8cfe7 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:10.477865 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 854/1733] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:44:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:44:10.478778 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c1612d-e9f3-4668-a129-ef4f8eee9ec6 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:10.486679 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-a1c1612d-e9f3-4668-a129-ef4f8eee9ec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:10.543057 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c55745-0ca9-415c-9498-c197ed1c9382 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24d5e2637b7043e08313909d5b06e329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:10.544133 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c55745-0ca9-415c-9498-c197ed1c9382 service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:10.545569 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c55745-0ca9-415c-9498-c197ed1c9382 service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:10.547161 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c55745-0ca9-415c-9498-c197ed1c9382 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:10.550302 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-60c55745-0ca9-415c-9498-c197ed1c9382 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:10.566839 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:44:10.647452 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c55745-0ca9-415c-9498-c197ed1c9382 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:10.649453 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c55745-0ca9-415c-9498-c197ed1c9382 service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:10.664747 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 880/1734] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:44:10 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 17:44:10.666184 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36949454-adbc-456b-bae2-72f26e42d0dc None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:10.666512 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36949454-adbc-456b-bae2-72f26e42d0dc None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:10.666790 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36949454-adbc-456b-bae2-72f26e42d0dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:10.698176 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c1612d-e9f3-4668-a129-ef4f8eee9ec6 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:10.708294 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d743b11-6541-4d8e-bc15-54f8864ced34 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f86c16179a70474a817be9fda33ece9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:10.709086 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d743b11-6541-4d8e-bc15-54f8864ced34 service nova] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:10.709505 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d743b11-6541-4d8e-bc15-54f8864ced34 service nova] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:10.709847 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d743b11-6541-4d8e-bc15-54f8864ced34 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:10.712821 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-4d743b11-6541-4d8e-bc15-54f8864ced34 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:10.727435 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-36949454-adbc-456b-bae2-72f26e42d0dc None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:44:10.732363 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-36949454-adbc-456b-bae2-72f26e42d0dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:10.764496 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 881/1735] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:44:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:44:10.770556 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2590b0-6639-4c65-a555-558c9df4487d None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:10.770957 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2590b0-6639-4c65-a555-558c9df4487d None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:10.771342 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2590b0-6639-4c65-a555-558c9df4487d None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:10.773098 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 882/1736] 10.222.0.42 () {54 vars in 960 bytes} [Thu Jun 16 17:44:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:44:10.776623 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2590b0-6639-4c65-a555-558c9df4487d None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:10.819736 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2590b0-6639-4c65-a555-558c9df4487d None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:10.837747 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d22b479-33dc-4e8b-8399-8f1606fbc026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:10.838418 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d22b479-33dc-4e8b-8399-8f1606fbc026 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:10.838708 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d22b479-33dc-4e8b-8399-8f1606fbc026 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:10.838991 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d22b479-33dc-4e8b-8399-8f1606fbc026 admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:10.839730 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d22b479-33dc-4e8b-8399-8f1606fbc026 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:44:10.839911 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d743b11-6541-4d8e-bc15-54f8864ced34 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:10.840066 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d22b479-33dc-4e8b-8399-8f1606fbc026 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:10.841692 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d22b479-33dc-4e8b-8399-8f1606fbc026 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:10.841946 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d743b11-6541-4d8e-bc15-54f8864ced34 service nova] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:10.856631 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 855/1737] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:44:10 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:44:10.860284 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cecf0b0-aa30-434e-98df-70feb109ae6f None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:10.860608 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cecf0b0-aa30-434e-98df-70feb109ae6f None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:10.863614 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cecf0b0-aa30-434e-98df-70feb109ae6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:10.886126 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 883/1738] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jun 16 17:44:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:44:10.887361 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d22b479-33dc-4e8b-8399-8f1606fbc026 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:10.896908 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-0d22b479-33dc-4e8b-8399-8f1606fbc026 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:10.923087 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d22b479-33dc-4e8b-8399-8f1606fbc026 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:10.935009 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6201e12d-b23a-45f5-99d0-82d9358d322a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24d5e2637b7043e08313909d5b06e329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:10.935677 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6201e12d-b23a-45f5-99d0-82d9358d322a service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:10.935968 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6201e12d-b23a-45f5-99d0-82d9358d322a service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:10.936249 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6201e12d-b23a-45f5-99d0-82d9358d322a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:10.942936 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-6201e12d-b23a-45f5-99d0-82d9358d322a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:10.952242 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-1cecf0b0-aa30-434e-98df-70feb109ae6f None None] MFA Rules not processed for user `0570d8d9508d4546ae897276f7d181d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:44:10.953719 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-1cecf0b0-aa30-434e-98df-70feb109ae6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:11.029366 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:44:11.092420 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 856/1739] 10.222.0.42 () {58 vars in 1057 bytes} [Thu Jun 16 17:44:10 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:44:11.095962 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cecf0b0-aa30-434e-98df-70feb109ae6f None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:11.098139 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6201e12d-b23a-45f5-99d0-82d9358d322a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:11.099611 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6201e12d-b23a-45f5-99d0-82d9358d322a service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:11.115218 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cecf0b0-aa30-434e-98df-70feb109ae6f None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:11.122835 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 884/1740] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:44:10 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:44:11.123322 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd44cb6-500c-4b3b-94ea-9ffb3c636bc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:11.125140 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd44cb6-500c-4b3b-94ea-9ffb3c636bc9 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:11.125407 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790f88f8-9bb6-4a92-9f13-b3dc09618e54 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:11.125555 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd44cb6-500c-4b3b-94ea-9ffb3c636bc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:11.125744 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790f88f8-9bb6-4a92-9f13-b3dc09618e54 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:11.125848 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd44cb6-500c-4b3b-94ea-9ffb3c636bc9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:11.126032 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790f88f8-9bb6-4a92-9f13-b3dc09618e54 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:11.126712 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd44cb6-500c-4b3b-94ea-9ffb3c636bc9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:11.129059 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 885/1741] 10.222.0.42 () {54 vars in 912 bytes} [Thu Jun 16 17:44:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:44:11.129189 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd44cb6-500c-4b3b-94ea-9ffb3c636bc9 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:11.129867 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790f88f8-9bb6-4a92-9f13-b3dc09618e54 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:11.165506 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790f88f8-9bb6-4a92-9f13-b3dc09618e54 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:11.172829 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 857/1742] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jun 16 17:44:11 2022] POST /identity/v3/projects => generated 401 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:44:11.174397 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d07dee-f791-49ea-94f6-e7b33e0adf1f None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:11.174691 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d07dee-f791-49ea-94f6-e7b33e0adf1f None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:11.175018 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d07dee-f791-49ea-94f6-e7b33e0adf1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:11.181525 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec17596-94a9-4a59-99f2-5926cbefaed7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0570d8d9508d4546ae897276f7d181d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:11.182726 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec17596-94a9-4a59-99f2-5926cbefaed7 service placement] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:11.183019 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec17596-94a9-4a59-99f2-5926cbefaed7 service placement] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:11.183304 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec17596-94a9-4a59-99f2-5926cbefaed7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:11.188430 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-8ec17596-94a9-4a59-99f2-5926cbefaed7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:11.216596 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-53d07dee-f791-49ea-94f6-e7b33e0adf1f None None] MFA Rules not processed for user `f94416103f6245888214a997b014f47a`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:44:11.216801 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec17596-94a9-4a59-99f2-5926cbefaed7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:11.219195 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-53d07dee-f791-49ea-94f6-e7b33e0adf1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:11.228845 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec17596-94a9-4a59-99f2-5926cbefaed7 service placement] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:11.242811 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 886/1743] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:44:11 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:44:11.243723 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec17596-94a9-4a59-99f2-5926cbefaed7 service placement] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:11.268667 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec17596-94a9-4a59-99f2-5926cbefaed7 service placement] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:11.309838 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 858/1744] 10.222.0.42 () {58 vars in 1009 bytes} [Thu Jun 16 17:44:11 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:44:11.368979 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d07dee-f791-49ea-94f6-e7b33e0adf1f None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:11.371014 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-53d07dee-f791-49ea-94f6-e7b33e0adf1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:11.397983 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d07dee-f791-49ea-94f6-e7b33e0adf1f None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:11.415973 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963ffa8a-e918-4270-ad62-f9970d28d4e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:11.416843 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963ffa8a-e918-4270-ad62-f9970d28d4e3 service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:11.417727 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963ffa8a-e918-4270-ad62-f9970d28d4e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:11.418450 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963ffa8a-e918-4270-ad62-f9970d28d4e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:11.419542 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce34b28f-92ca-470f-a8ab-1da062dd6620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:11.420243 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce34b28f-92ca-470f-a8ab-1da062dd6620 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:11.420561 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce34b28f-92ca-470f-a8ab-1da062dd6620 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:11.422407 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce34b28f-92ca-470f-a8ab-1da062dd6620 admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:11.427470 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce34b28f-92ca-470f-a8ab-1da062dd6620 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:11.435846 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963ffa8a-e918-4270-ad62-f9970d28d4e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:11.438881 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963ffa8a-e918-4270-ad62-f9970d28d4e3 service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:11.446009 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce34b28f-92ca-470f-a8ab-1da062dd6620 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:11.464899 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 859/1745] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:44:11 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:44:11.491548 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:44:11.503678 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 887/1746] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jun 16 17:44:11 2022] POST /identity/v3/users => generated 301 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:44:11.508803 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963ffa8a-e918-4270-ad62-f9970d28d4e3 service neutron] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:11.552574 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963ffa8a-e918-4270-ad62-f9970d28d4e3 service neutron] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:11.581504 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57136f3b-58f8-4c92-9f7d-4c810c650de8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:11.585189 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57136f3b-58f8-4c92-9f7d-4c810c650de8 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:11.586349 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57136f3b-58f8-4c92-9f7d-4c810c650de8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:11.586836 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57136f3b-58f8-4c92-9f7d-4c810c650de8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:11.587887 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57136f3b-58f8-4c92-9f7d-4c810c650de8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:11.589947 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57136f3b-58f8-4c92-9f7d-4c810c650de8 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:11.607237 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 860/1747] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:44:11 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:44:11.611259 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce34b28f-92ca-470f-a8ab-1da062dd6620 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:11.627230 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce34b28f-92ca-470f-a8ab-1da062dd6620 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:11.633543 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a9f5a9-3b89-47b0-9fb4-979880c4b051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:11.634461 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a9f5a9-3b89-47b0-9fb4-979880c4b051 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:11.634793 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a9f5a9-3b89-47b0-9fb4-979880c4b051 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:11.635206 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a9f5a9-3b89-47b0-9fb4-979880c4b051 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:11.636211 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a9f5a9-3b89-47b0-9fb4-979880c4b051 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:11.638080 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a9f5a9-3b89-47b0-9fb4-979880c4b051 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:11.655219 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 888/1748] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:44:11 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:44:11.656784 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57136f3b-58f8-4c92-9f7d-4c810c650de8 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:11.670482 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57136f3b-58f8-4c92-9f7d-4c810c650de8 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:11.681556 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546ddf3a-c03b-488a-93ca-7652984d478b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:11.682471 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546ddf3a-c03b-488a-93ca-7652984d478b admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:11.682752 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546ddf3a-c03b-488a-93ca-7652984d478b admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:11.683029 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546ddf3a-c03b-488a-93ca-7652984d478b admin admin] PATH_INFO: `/v3/projects/7a38ccb4f65243a9a7253ebe3c4d93fa/users/a0981ad76c9741f38c62553efcd23c8c/roles/0be2075946f54c8485f1c275f746478a` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:11.765927 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546ddf3a-c03b-488a-93ca-7652984d478b admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a38ccb4f65243a9a7253ebe3c4d93fa, user_id=a0981ad76c9741f38c62553efcd23c8c, role_id=0be2075946f54c8485f1c275f746478a)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:11.774149 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546ddf3a-c03b-488a-93ca-7652984d478b admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:11.804303 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 861/1749] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:44:11 2022] PUT /identity/v3/projects/7a38ccb4f65243a9a7253ebe3c4d93fa/users/a0981ad76c9741f38c62553efcd23c8c/roles/0be2075946f54c8485f1c275f746478a => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:44:11.817617 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c4975b-3461-4e6e-880f-25a9217e7659 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:11.817785 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c4975b-3461-4e6e-880f-25a9217e7659 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:11.818057 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c4975b-3461-4e6e-880f-25a9217e7659 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:11.941141 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-a1c4975b-3461-4e6e-880f-25a9217e7659 None None] MFA Rules not processed for user `a0981ad76c9741f38c62553efcd23c8c`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:44:11.947264 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-a1c4975b-3461-4e6e-880f-25a9217e7659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:12.165723 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 889/1750] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:44:11 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 346 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:44:12.166985 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546ddf3a-c03b-488a-93ca-7652984d478b admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:12.392489 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546ddf3a-c03b-488a-93ca-7652984d478b admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:12.411477 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f37d74-e8ad-4b04-8456-4587d291da51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:12.412253 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f37d74-e8ad-4b04-8456-4587d291da51 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:12.412529 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f37d74-e8ad-4b04-8456-4587d291da51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:12.413065 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f37d74-e8ad-4b04-8456-4587d291da51 admin admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:12.420524 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f37d74-e8ad-4b04-8456-4587d291da51 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:12.420524 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f37d74-e8ad-4b04-8456-4587d291da51 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:12.485414 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:44:12.525036 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 862/1751] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jun 16 17:44:12 2022] POST /identity/v3/projects => generated 399 bytes in 358 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:44:12.525388 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c4975b-3461-4e6e-880f-25a9217e7659 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:12.566664 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c4975b-3461-4e6e-880f-25a9217e7659 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:12.717979 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44435430-c016-4810-a74f-7af4bbcd89aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:12.718949 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44435430-c016-4810-a74f-7af4bbcd89aa admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:12.719366 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44435430-c016-4810-a74f-7af4bbcd89aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:12.719786 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44435430-c016-4810-a74f-7af4bbcd89aa admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:12.720785 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44435430-c016-4810-a74f-7af4bbcd89aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:12.723135 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44435430-c016-4810-a74f-7af4bbcd89aa admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:12.763146 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 890/1752] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jun 16 17:44:12 2022] POST /identity/v3/users => generated 306 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:44:12.767581 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f37d74-e8ad-4b04-8456-4587d291da51 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:12.787457 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f37d74-e8ad-4b04-8456-4587d291da51 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:12.818039 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20592fc-b70b-4c6f-88f0-300e7cb8f336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:12.826142 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20592fc-b70b-4c6f-88f0-300e7cb8f336 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:12.826793 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20592fc-b70b-4c6f-88f0-300e7cb8f336 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:12.828188 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20592fc-b70b-4c6f-88f0-300e7cb8f336 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:12.833620 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20592fc-b70b-4c6f-88f0-300e7cb8f336 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:12.835436 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20592fc-b70b-4c6f-88f0-300e7cb8f336 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:12.884718 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44435430-c016-4810-a74f-7af4bbcd89aa admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:12.885298 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 863/1753] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:44:12 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 17:44:12.911643 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44435430-c016-4810-a74f-7af4bbcd89aa admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:12.941887 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850a0e75-3369-4c6f-91c1-ae914370d784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:12.945960 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850a0e75-3369-4c6f-91c1-ae914370d784 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:12.946571 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850a0e75-3369-4c6f-91c1-ae914370d784 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:12.948629 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850a0e75-3369-4c6f-91c1-ae914370d784 admin admin] PATH_INFO: `/v3/projects/69d9dcc42f874a8db5889ef9143a851d/users/59dfac930e394e368caf5d150cbd6a63/roles/3ca2dcbbd29d4a31a3ab78f2299bc239` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:12.976531 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef50708a-40f6-448d-ae48-e7682769bf38 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:12.977011 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef50708a-40f6-448d-ae48-e7682769bf38 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:12.977536 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef50708a-40f6-448d-ae48-e7682769bf38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:13.021854 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-ef50708a-40f6-448d-ae48-e7682769bf38 None None] MFA Rules not processed for user `7b078c6bfca04de790e7b2e45739033e`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:44:13.025736 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-ef50708a-40f6-448d-ae48-e7682769bf38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:13.030269 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850a0e75-3369-4c6f-91c1-ae914370d784 admin admin] RBAC: Authorizing `identity:create_grant(project_id=69d9dcc42f874a8db5889ef9143a851d, user_id=59dfac930e394e368caf5d150cbd6a63, role_id=3ca2dcbbd29d4a31a3ab78f2299bc239)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:13.033751 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850a0e75-3369-4c6f-91c1-ae914370d784 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:13.086105 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 891/1754] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:44:12 2022] PUT /identity/v3/projects/69d9dcc42f874a8db5889ef9143a851d/users/59dfac930e394e368caf5d150cbd6a63/roles/3ca2dcbbd29d4a31a3ab78f2299bc239 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:44:13.087262 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850a0e75-3369-4c6f-91c1-ae914370d784 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:13.224644 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 864/1755] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:44:12 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:44:13.239836 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:44:13.290720 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850a0e75-3369-4c6f-91c1-ae914370d784 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:13.323482 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc10963-b802-49c0-88f1-559204c196b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:13.324255 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc10963-b802-49c0-88f1-559204c196b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:13.324652 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc10963-b802-49c0-88f1-559204c196b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:13.324786 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc10963-b802-49c0-88f1-559204c196b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:13.325644 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc10963-b802-49c0-88f1-559204c196b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:13.327397 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc10963-b802-49c0-88f1-559204c196b6 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:13.377256 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 892/1756] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:44:13 2022] GET /identity/v3/roles => generated 1420 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:44:13.381431 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef50708a-40f6-448d-ae48-e7682769bf38 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:13.397331 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef50708a-40f6-448d-ae48-e7682769bf38 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:13.410532 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8c063d-dcfa-4d2e-92bd-981175c32482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:13.411188 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8c063d-dcfa-4d2e-92bd-981175c32482 admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:13.411536 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8c063d-dcfa-4d2e-92bd-981175c32482 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:13.411795 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8c063d-dcfa-4d2e-92bd-981175c32482 admin admin] PATH_INFO: `/v3/projects/69d9dcc42f874a8db5889ef9143a851d/users/59dfac930e394e368caf5d150cbd6a63/roles/3ca2dcbbd29d4a31a3ab78f2299bc239` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:13.418180 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8c063d-dcfa-4d2e-92bd-981175c32482 admin admin] RBAC: Authorizing `identity:create_grant(project_id=69d9dcc42f874a8db5889ef9143a851d, user_id=59dfac930e394e368caf5d150cbd6a63, role_id=3ca2dcbbd29d4a31a3ab78f2299bc239)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:13.422230 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8c063d-dcfa-4d2e-92bd-981175c32482 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:13.471031 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 865/1757] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:44:13 2022] PUT /identity/v3/projects/69d9dcc42f874a8db5889ef9143a851d/users/59dfac930e394e368caf5d150cbd6a63/roles/3ca2dcbbd29d4a31a3ab78f2299bc239 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:44:13.484109 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95180ed1-e5b8-4ca8-8bde-cc04fcdc8342 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:13.484430 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95180ed1-e5b8-4ca8-8bde-cc04fcdc8342 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:13.484697 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95180ed1-e5b8-4ca8-8bde-cc04fcdc8342 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:13.578484 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-95180ed1-e5b8-4ca8-8bde-cc04fcdc8342 None None] MFA Rules not processed for user `59dfac930e394e368caf5d150cbd6a63`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:44:13.583865 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-95180ed1-e5b8-4ca8-8bde-cc04fcdc8342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:13.816306 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 893/1758] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:44:13 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 329 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:44:13.823602 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8c063d-dcfa-4d2e-92bd-981175c32482 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:13.826173 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-eb8c063d-dcfa-4d2e-92bd-981175c32482 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:13.962207 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1dccd2-e0cc-4849-b42d-daf57fc98a34 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:13.963080 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1dccd2-e0cc-4849-b42d-daf57fc98a34 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:13.963667 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1dccd2-e0cc-4849-b42d-daf57fc98a34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:14.008616 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-fa1dccd2-e0cc-4849-b42d-daf57fc98a34 None None] MFA Rules not processed for user `8fff53cb0d084d9284825488f6843ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:44:14.011061 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-fa1dccd2-e0cc-4849-b42d-daf57fc98a34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:14.093795 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8c063d-dcfa-4d2e-92bd-981175c32482 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:14.117175 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9203fac0-3ca2-47ca-b33b-c2d42c57970e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24d5e2637b7043e08313909d5b06e329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:14.117860 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9203fac0-3ca2-47ca-b33b-c2d42c57970e service cinder] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:14.118140 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9203fac0-3ca2-47ca-b33b-c2d42c57970e service cinder] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:14.118424 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9203fac0-3ca2-47ca-b33b-c2d42c57970e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:14.131494 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 894/1759] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:44:13 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:44:14.141225 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9203fac0-3ca2-47ca-b33b-c2d42c57970e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:14.142819 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9203fac0-3ca2-47ca-b33b-c2d42c57970e service cinder] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:14.161985 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 866/1760] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:44:13 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:44:14.322726 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1dccd2-e0cc-4849-b42d-daf57fc98a34 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:14.350183 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1dccd2-e0cc-4849-b42d-daf57fc98a34 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:14.373311 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1622cc12-804a-4232-bfea-4e91e9665774 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24d5e2637b7043e08313909d5b06e329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:14.374534 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1622cc12-804a-4232-bfea-4e91e9665774 service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:14.374823 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1622cc12-804a-4232-bfea-4e91e9665774 service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:14.375108 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1622cc12-804a-4232-bfea-4e91e9665774 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:14.701541 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1622cc12-804a-4232-bfea-4e91e9665774 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:14.703771 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1622cc12-804a-4232-bfea-4e91e9665774 service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:14.722124 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 895/1761] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:44:14 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:44:14.778684 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9203fac0-3ca2-47ca-b33b-c2d42c57970e service cinder] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:14.780958 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-9203fac0-3ca2-47ca-b33b-c2d42c57970e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:15.053373 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9203fac0-3ca2-47ca-b33b-c2d42c57970e service cinder] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:15.072234 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88306b7-558e-4f0c-be8b-5e9836faeec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:15.073411 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88306b7-558e-4f0c-be8b-5e9836faeec4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:15.074743 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88306b7-558e-4f0c-be8b-5e9836faeec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:15.075172 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88306b7-558e-4f0c-be8b-5e9836faeec4 admin admin] PATH_INFO: `/v3/users/2097f78edf1446108d0b7ba2943048e3` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:15.078044 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88306b7-558e-4f0c-be8b-5e9836faeec4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2097f78edf1446108d0b7ba2943048e3)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:15.086114 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88306b7-558e-4f0c-be8b-5e9836faeec4 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:15.179777 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:44:15.251221 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-a88306b7-558e-4f0c-be8b-5e9836faeec4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2097f78edf1446108d0b7ba2943048e3'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:15.253023 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-a88306b7-558e-4f0c-be8b-5e9836faeec4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2097f78edf1446108d0b7ba2943048e3'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:15.279974 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-a88306b7-558e-4f0c-be8b-5e9836faeec4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2097f78edf1446108d0b7ba2943048e3'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:15.282913 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-a88306b7-558e-4f0c-be8b-5e9836faeec4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2097f78edf1446108d0b7ba2943048e3'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:15.362550 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-a88306b7-558e-4f0c-be8b-5e9836faeec4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2097f78edf1446108d0b7ba2943048e3'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:15.422423 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 867/1762] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jun 16 17:44:14 2022] DELETE /identity/v3/users/2097f78edf1446108d0b7ba2943048e3 => generated 0 bytes in 640 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:44:15.424348 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065b1898-25c3-42da-96da-ea3d6eb19183 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:15.424685 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065b1898-25c3-42da-96da-ea3d6eb19183 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:15.424970 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065b1898-25c3-42da-96da-ea3d6eb19183 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:15.490950 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-065b1898-25c3-42da-96da-ea3d6eb19183 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:44:15.491115 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d84e55c-31b4-4733-be19-e216b9461aa5 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:15.491338 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d84e55c-31b4-4733-be19-e216b9461aa5 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:15.492449 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d84e55c-31b4-4733-be19-e216b9461aa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:15.492694 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-065b1898-25c3-42da-96da-ea3d6eb19183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:15.507938 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:44:15.549578 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-1d84e55c-31b4-4733-be19-e216b9461aa5 None None] MFA Rules not processed for user `268c5947a265492bb7a8f181f9f5c504`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:44:15.557545 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-1d84e55c-31b4-4733-be19-e216b9461aa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:15.722594 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 896/1763] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:44:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:44:15.812608 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065b1898-25c3-42da-96da-ea3d6eb19183 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:15.817764 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-065b1898-25c3-42da-96da-ea3d6eb19183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:15.819435 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 868/1764] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:44:15 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 329 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:44:15.992883 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065b1898-25c3-42da-96da-ea3d6eb19183 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:16.004786 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c84b96-34af-4edb-9fb5-a166d4004159 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:16.005627 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c84b96-34af-4edb-9fb5-a166d4004159 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:16.006698 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c84b96-34af-4edb-9fb5-a166d4004159 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:16.007166 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c84b96-34af-4edb-9fb5-a166d4004159 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:16.031133 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c84b96-34af-4edb-9fb5-a166d4004159 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:16.032515 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c84b96-34af-4edb-9fb5-a166d4004159 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:16.058232 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 897/1765] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:44:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 17:44:16.838993 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d84e55c-31b4-4733-be19-e216b9461aa5 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:16.840702 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-1d84e55c-31b4-4733-be19-e216b9461aa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:16.865499 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d84e55c-31b4-4733-be19-e216b9461aa5 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:16.907263 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d7fee4-8ab3-401c-8306-7ece7e9cd001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:16.908317 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d7fee4-8ab3-401c-8306-7ece7e9cd001 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:16.908808 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d7fee4-8ab3-401c-8306-7ece7e9cd001 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:16.909240 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d7fee4-8ab3-401c-8306-7ece7e9cd001 admin admin] PATH_INFO: `/v3/projects/75dc17395d7644708d5d6e808b30830c` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:16.910627 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d7fee4-8ab3-401c-8306-7ece7e9cd001 admin admin] RBAC: Authorizing `identity:delete_project(project_id=75dc17395d7644708d5d6e808b30830c)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:16.912739 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d7fee4-8ab3-401c-8306-7ece7e9cd001 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:17.016750 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-01d7fee4-8ab3-401c-8306-7ece7e9cd001 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75dc17395d7644708d5d6e808b30830c', 'request_id': 'req-01d7fee4-8ab3-401c-8306-7ece7e9cd001'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:17.050110 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-01d7fee4-8ab3-401c-8306-7ece7e9cd001 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75dc17395d7644708d5d6e808b30830c', 'request_id': 'req-01d7fee4-8ab3-401c-8306-7ece7e9cd001'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:17.061474 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-01d7fee4-8ab3-401c-8306-7ece7e9cd001 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75dc17395d7644708d5d6e808b30830c', 'request_id': 'req-01d7fee4-8ab3-401c-8306-7ece7e9cd001'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:17.073970 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-01d7fee4-8ab3-401c-8306-7ece7e9cd001 admin admin] The token cache is being invalidate because project 75dc17395d7644708d5d6e808b30830c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80357) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:44:17.074241 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-01d7fee4-8ab3-401c-8306-7ece7e9cd001 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:17.076729 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-01d7fee4-8ab3-401c-8306-7ece7e9cd001 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:17.079962 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 869/1766] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jun 16 17:44:16 2022] DELETE /identity/v3/projects/75dc17395d7644708d5d6e808b30830c => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:44:17.084020 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c84b96-34af-4edb-9fb5-a166d4004159 service neutron] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:17.113471 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c84b96-34af-4edb-9fb5-a166d4004159 service neutron] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:17.270089 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9ec4ac-c381-49b7-8ef9-6aaf2987fc1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:17.270976 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9ec4ac-c381-49b7-8ef9-6aaf2987fc1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:17.271414 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9ec4ac-c381-49b7-8ef9-6aaf2987fc1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:17.271834 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9ec4ac-c381-49b7-8ef9-6aaf2987fc1e admin admin] PATH_INFO: `/v3/users/bdc09b9e889d483680a4f44f166af074` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:17.276477 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9ec4ac-c381-49b7-8ef9-6aaf2987fc1e admin admin] RBAC: Authorizing `identity:delete_user(user_id=bdc09b9e889d483680a4f44f166af074)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:17.278616 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9ec4ac-c381-49b7-8ef9-6aaf2987fc1e admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:17.383792 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-eb9ec4ac-c381-49b7-8ef9-6aaf2987fc1e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bdc09b9e889d483680a4f44f166af074'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:17.398261 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:44:17.433504 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-eb9ec4ac-c381-49b7-8ef9-6aaf2987fc1e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bdc09b9e889d483680a4f44f166af074'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:17.434285 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-eb9ec4ac-c381-49b7-8ef9-6aaf2987fc1e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bdc09b9e889d483680a4f44f166af074'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:17.458086 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-eb9ec4ac-c381-49b7-8ef9-6aaf2987fc1e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bdc09b9e889d483680a4f44f166af074'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:17.487854 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-eb9ec4ac-c381-49b7-8ef9-6aaf2987fc1e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bdc09b9e889d483680a4f44f166af074'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:17.496173 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 898/1767] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jun 16 17:44:17 2022] DELETE /identity/v3/users/bdc09b9e889d483680a4f44f166af074 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:44:18.074011 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b40a45f-424f-4cc7-a29d-dc9ee5de89fa None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:18.074929 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b40a45f-424f-4cc7-a29d-dc9ee5de89fa None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:18.075195 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b40a45f-424f-4cc7-a29d-dc9ee5de89fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:18.109860 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-8b40a45f-424f-4cc7-a29d-dc9ee5de89fa None None] MFA Rules not processed for user `71443c30b4554719b5549493bf3cc00c`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:44:18.111271 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-8b40a45f-424f-4cc7-a29d-dc9ee5de89fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:18.154171 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:44:18.237302 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 870/1768] 10.222.0.42 () {58 vars in 1009 bytes} [Thu Jun 16 17:44:18 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:44:18.445032 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9ec4ac-c381-49b7-8ef9-6aaf2987fc1e admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:18.447108 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-eb9ec4ac-c381-49b7-8ef9-6aaf2987fc1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:18.554740 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9ec4ac-c381-49b7-8ef9-6aaf2987fc1e admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:18.563347 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb4cbdb-2596-4d0a-a365-276cbcde7240 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47b7a9cd615c41d6983a0007b475038c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:18.564833 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb4cbdb-2596-4d0a-a365-276cbcde7240 service swift] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:18.565375 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb4cbdb-2596-4d0a-a365-276cbcde7240 service swift] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:18.565862 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb4cbdb-2596-4d0a-a365-276cbcde7240 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:18.588959 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb4cbdb-2596-4d0a-a365-276cbcde7240 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:18.591372 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb4cbdb-2596-4d0a-a365-276cbcde7240 service swift] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:18.610927 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b40a45f-424f-4cc7-a29d-dc9ee5de89fa None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:18.616531 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-8b40a45f-424f-4cc7-a29d-dc9ee5de89fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:18.644860 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 899/1769] 10.222.0.42 () {60 vars in 1384 bytes} [Thu Jun 16 17:44:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 17:44:18.872284 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b40a45f-424f-4cc7-a29d-dc9ee5de89fa None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:18.896228 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e51cce3-9eae-43f5-bbfd-46e1192ce89d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:18.897530 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e51cce3-9eae-43f5-bbfd-46e1192ce89d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:18.901838 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e51cce3-9eae-43f5-bbfd-46e1192ce89d admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:18.903160 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e51cce3-9eae-43f5-bbfd-46e1192ce89d admin admin] PATH_INFO: `/v3/projects/319de500a1894a418ebab7257eeb66c1` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:18.906738 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e51cce3-9eae-43f5-bbfd-46e1192ce89d admin admin] RBAC: Authorizing `identity:delete_project(project_id=319de500a1894a418ebab7257eeb66c1)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:18.913036 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e51cce3-9eae-43f5-bbfd-46e1192ce89d admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:19.121657 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-5e51cce3-9eae-43f5-bbfd-46e1192ce89d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '319de500a1894a418ebab7257eeb66c1', 'request_id': 'req-5e51cce3-9eae-43f5-bbfd-46e1192ce89d'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:19.151819 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-5e51cce3-9eae-43f5-bbfd-46e1192ce89d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '319de500a1894a418ebab7257eeb66c1', 'request_id': 'req-5e51cce3-9eae-43f5-bbfd-46e1192ce89d'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:19.166810 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-5e51cce3-9eae-43f5-bbfd-46e1192ce89d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '319de500a1894a418ebab7257eeb66c1', 'request_id': 'req-5e51cce3-9eae-43f5-bbfd-46e1192ce89d'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:19.204595 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-5e51cce3-9eae-43f5-bbfd-46e1192ce89d admin admin] The token cache is being invalidate because project 319de500a1894a418ebab7257eeb66c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80357) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:44:19.204861 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-5e51cce3-9eae-43f5-bbfd-46e1192ce89d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:19.220919 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-5e51cce3-9eae-43f5-bbfd-46e1192ce89d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:19.224023 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 871/1770] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jun 16 17:44:18 2022] DELETE /identity/v3/projects/319de500a1894a418ebab7257eeb66c1 => generated 0 bytes in 614 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:44:19.229332 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff67754-c23b-42c9-8e30-c50164b88185 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:19.230480 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff67754-c23b-42c9-8e30-c50164b88185 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:19.230988 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff67754-c23b-42c9-8e30-c50164b88185 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:19.295968 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-8ff67754-c23b-42c9-8e30-c50164b88185 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:44:19.297463 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-8ff67754-c23b-42c9-8e30-c50164b88185 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:19.582476 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 900/1771] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:44:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 354 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:44:19.591945 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bf0ea0-21ff-4c61-954a-e163ca9744ce None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:19.592905 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bf0ea0-21ff-4c61-954a-e163ca9744ce None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:19.596449 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bf0ea0-21ff-4c61-954a-e163ca9744ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:19.677870 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-19bf0ea0-21ff-4c61-954a-e163ca9744ce None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:44:19.680524 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-19bf0ea0-21ff-4c61-954a-e163ca9744ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:19.711566 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 872/1772] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:44:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:44:19.714241 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff67754-c23b-42c9-8e30-c50164b88185 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:19.736343 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff67754-c23b-42c9-8e30-c50164b88185 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:19.747626 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbe09c9-bd0b-4152-817b-9a699d2f6917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:19.748567 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbe09c9-bd0b-4152-817b-9a699d2f6917 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:19.749586 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbe09c9-bd0b-4152-817b-9a699d2f6917 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:19.750029 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbe09c9-bd0b-4152-817b-9a699d2f6917 admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:19.751042 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbe09c9-bd0b-4152-817b-9a699d2f6917 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:44:19.751632 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbe09c9-bd0b-4152-817b-9a699d2f6917 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:19.758246 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbe09c9-bd0b-4152-817b-9a699d2f6917 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:19.804227 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 901/1773] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jun 16 17:44:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:44:19.808674 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bf0ea0-21ff-4c61-954a-e163ca9744ce None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:19.824190 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bf0ea0-21ff-4c61-954a-e163ca9744ce None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:19.839861 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a72516c-9a18-401b-8b78-db57382c5930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:19.845913 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a72516c-9a18-401b-8b78-db57382c5930 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:19.846219 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a72516c-9a18-401b-8b78-db57382c5930 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:19.846483 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a72516c-9a18-401b-8b78-db57382c5930 admin admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:19.847924 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a72516c-9a18-401b-8b78-db57382c5930 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:19.849962 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a72516c-9a18-401b-8b78-db57382c5930 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:19.985537 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 873/1774] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jun 16 17:44:19 2022] POST /identity/v3/projects => generated 385 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:44:19.988521 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbe09c9-bd0b-4152-817b-9a699d2f6917 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:20.023878 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbe09c9-bd0b-4152-817b-9a699d2f6917 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:20.117587 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:44:20.171976 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3d130a-fc74-4799-a535-3a4f4c27a3b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:20.175049 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3d130a-fc74-4799-a535-3a4f4c27a3b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:20.175417 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3d130a-fc74-4799-a535-3a4f4c27a3b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:20.187208 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3d130a-fc74-4799-a535-3a4f4c27a3b4 admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:20.198565 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3d130a-fc74-4799-a535-3a4f4c27a3b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:20.209289 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3d130a-fc74-4799-a535-3a4f4c27a3b4 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:20.272421 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 902/1775] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jun 16 17:44:19 2022] POST /identity/v3/users => generated 293 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:44:20.274834 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a72516c-9a18-401b-8b78-db57382c5930 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:20.292675 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a72516c-9a18-401b-8b78-db57382c5930 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:20.317112 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5789f113-dee7-4f53-a24d-a9018320e52d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:20.317813 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5789f113-dee7-4f53-a24d-a9018320e52d admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:20.318099 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5789f113-dee7-4f53-a24d-a9018320e52d admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:20.326790 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5789f113-dee7-4f53-a24d-a9018320e52d admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:20.328407 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5789f113-dee7-4f53-a24d-a9018320e52d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:20.330315 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5789f113-dee7-4f53-a24d-a9018320e52d admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:20.353758 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 874/1776] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:44:20 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:44:20.359642 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3d130a-fc74-4799-a535-3a4f4c27a3b4 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:20.381201 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3d130a-fc74-4799-a535-3a4f4c27a3b4 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:20.390496 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f67345-6f32-4732-877c-c749c1d9a935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:20.391606 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f67345-6f32-4732-877c-c749c1d9a935 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:20.392145 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f67345-6f32-4732-877c-c749c1d9a935 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:20.392557 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f67345-6f32-4732-877c-c749c1d9a935 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:20.393681 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f67345-6f32-4732-877c-c749c1d9a935 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:20.396902 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f67345-6f32-4732-877c-c749c1d9a935 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:20.416792 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 903/1777] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:44:20 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:44:20.421398 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5789f113-dee7-4f53-a24d-a9018320e52d admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:20.435151 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5789f113-dee7-4f53-a24d-a9018320e52d admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:20.442375 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7031f00b-d069-4af6-a86f-ee82e656c086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:20.443449 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7031f00b-d069-4af6-a86f-ee82e656c086 admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:20.443809 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7031f00b-d069-4af6-a86f-ee82e656c086 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:20.444101 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7031f00b-d069-4af6-a86f-ee82e656c086 admin admin] PATH_INFO: `/v3/projects/079924db14e4476b8650b08bae2c9dbd/users/eec94d3f43be45a38c491371148ef3d8/roles/0be2075946f54c8485f1c275f746478a` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:20.488917 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7031f00b-d069-4af6-a86f-ee82e656c086 admin admin] RBAC: Authorizing `identity:create_grant(project_id=079924db14e4476b8650b08bae2c9dbd, user_id=eec94d3f43be45a38c491371148ef3d8, role_id=0be2075946f54c8485f1c275f746478a)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:20.492626 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7031f00b-d069-4af6-a86f-ee82e656c086 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:20.530547 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:44:20.539672 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 875/1778] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:44:20 2022] PUT /identity/v3/projects/079924db14e4476b8650b08bae2c9dbd/users/eec94d3f43be45a38c491371148ef3d8/roles/0be2075946f54c8485f1c275f746478a => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:44:20.551048 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87291bfe-251a-430c-bdde-e8085616d0d2 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:20.551344 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87291bfe-251a-430c-bdde-e8085616d0d2 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:20.552872 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87291bfe-251a-430c-bdde-e8085616d0d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:20.597165 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-87291bfe-251a-430c-bdde-e8085616d0d2 None None] MFA Rules not processed for user `eec94d3f43be45a38c491371148ef3d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:44:20.598809 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-87291bfe-251a-430c-bdde-e8085616d0d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:20.845206 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 904/1779] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:44:20 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:44:20.856486 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7031f00b-d069-4af6-a86f-ee82e656c086 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:20.859113 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-7031f00b-d069-4af6-a86f-ee82e656c086 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:20.993362 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7031f00b-d069-4af6-a86f-ee82e656c086 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:21.002815 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87e2c6d-fab5-437b-aea8-158d0114ae39 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24d5e2637b7043e08313909d5b06e329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:21.004533 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87e2c6d-fab5-437b-aea8-158d0114ae39 service cinder] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:21.006768 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87e2c6d-fab5-437b-aea8-158d0114ae39 service cinder] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:21.006768 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87e2c6d-fab5-437b-aea8-158d0114ae39 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:21.056185 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87e2c6d-fab5-437b-aea8-158d0114ae39 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:21.057711 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87e2c6d-fab5-437b-aea8-158d0114ae39 service cinder] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:21.106391 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 876/1780] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:44:20 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:44:21.131490 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87291bfe-251a-430c-bdde-e8085616d0d2 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:21.135839 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf34b8b-b6b8-454b-9bd9-e084109d8e1f None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:21.136282 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf34b8b-b6b8-454b-9bd9-e084109d8e1f None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:21.136807 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf34b8b-b6b8-454b-9bd9-e084109d8e1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:21.197800 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-4bf34b8b-b6b8-454b-9bd9-e084109d8e1f None None] MFA Rules not processed for user `f94416103f6245888214a997b014f47a`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:44:21.199497 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-4bf34b8b-b6b8-454b-9bd9-e084109d8e1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:21.292339 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 877/1781] 10.222.0.15 () {58 vars in 1015 bytes} [Thu Jun 16 17:44:21 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:44:21.398570 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87291bfe-251a-430c-bdde-e8085616d0d2 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:21.420507 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313fd887-9596-46d7-97c0-efc9362b30dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:21.420507 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313fd887-9596-46d7-97c0-efc9362b30dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:21.420507 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313fd887-9596-46d7-97c0-efc9362b30dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:21.420507 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313fd887-9596-46d7-97c0-efc9362b30dd admin admin] PATH_INFO: `/v3/users/eec94d3f43be45a38c491371148ef3d8` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:21.424239 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313fd887-9596-46d7-97c0-efc9362b30dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=eec94d3f43be45a38c491371148ef3d8)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:21.428490 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313fd887-9596-46d7-97c0-efc9362b30dd admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:21.573364 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-313fd887-9596-46d7-97c0-efc9362b30dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eec94d3f43be45a38c491371148ef3d8'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:21.590756 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:44:21.602524 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-313fd887-9596-46d7-97c0-efc9362b30dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eec94d3f43be45a38c491371148ef3d8'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:21.604745 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-313fd887-9596-46d7-97c0-efc9362b30dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eec94d3f43be45a38c491371148ef3d8'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:21.620844 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-313fd887-9596-46d7-97c0-efc9362b30dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eec94d3f43be45a38c491371148ef3d8'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:21.634619 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-313fd887-9596-46d7-97c0-efc9362b30dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eec94d3f43be45a38c491371148ef3d8'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:21.653174 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 905/1782] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jun 16 17:44:21 2022] DELETE /identity/v3/users/eec94d3f43be45a38c491371148ef3d8 => generated 0 bytes in 519 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:44:21.657605 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2060d863-5a76-4737-99fb-4f109fa3edc5 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:21.657988 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2060d863-5a76-4737-99fb-4f109fa3edc5 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:21.658541 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2060d863-5a76-4737-99fb-4f109fa3edc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:21.680410 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-2060d863-5a76-4737-99fb-4f109fa3edc5 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:44:21.681884 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-2060d863-5a76-4737-99fb-4f109fa3edc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:21.777576 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 878/1783] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:44:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:44:21.787476 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313fd887-9596-46d7-97c0-efc9362b30dd admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:21.789623 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-313fd887-9596-46d7-97c0-efc9362b30dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:21.889410 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313fd887-9596-46d7-97c0-efc9362b30dd admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:21.893572 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f325e660-4bfe-40ff-bbf3-3f331d23e780 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:21.894255 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f325e660-4bfe-40ff-bbf3-3f331d23e780 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:21.894536 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f325e660-4bfe-40ff-bbf3-3f331d23e780 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:21.894812 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f325e660-4bfe-40ff-bbf3-3f331d23e780 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:21.908954 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f325e660-4bfe-40ff-bbf3-3f331d23e780 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:21.910707 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f325e660-4bfe-40ff-bbf3-3f331d23e780 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:21.923686 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 906/1784] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:44:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:44:22.270152 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2060d863-5a76-4737-99fb-4f109fa3edc5 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:22.271670 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-2060d863-5a76-4737-99fb-4f109fa3edc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:22.284781 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2060d863-5a76-4737-99fb-4f109fa3edc5 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:22.291518 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb4eab5-2d1d-4d4e-a4b8-20288e4bed69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:22.292732 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb4eab5-2d1d-4d4e-a4b8-20288e4bed69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:22.294202 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb4eab5-2d1d-4d4e-a4b8-20288e4bed69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:22.294708 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb4eab5-2d1d-4d4e-a4b8-20288e4bed69 admin admin] PATH_INFO: `/v3/projects/079924db14e4476b8650b08bae2c9dbd` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:22.297870 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb4eab5-2d1d-4d4e-a4b8-20288e4bed69 admin admin] RBAC: Authorizing `identity:delete_project(project_id=079924db14e4476b8650b08bae2c9dbd)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:22.300436 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb4eab5-2d1d-4d4e-a4b8-20288e4bed69 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:22.373320 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:44:22.383519 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-ccb4eab5-2d1d-4d4e-a4b8-20288e4bed69 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '079924db14e4476b8650b08bae2c9dbd', 'request_id': 'req-ccb4eab5-2d1d-4d4e-a4b8-20288e4bed69'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:22.394478 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-ccb4eab5-2d1d-4d4e-a4b8-20288e4bed69 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '079924db14e4476b8650b08bae2c9dbd', 'request_id': 'req-ccb4eab5-2d1d-4d4e-a4b8-20288e4bed69'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:22.402528 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-ccb4eab5-2d1d-4d4e-a4b8-20288e4bed69 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '079924db14e4476b8650b08bae2c9dbd', 'request_id': 'req-ccb4eab5-2d1d-4d4e-a4b8-20288e4bed69'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:22.417082 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-ccb4eab5-2d1d-4d4e-a4b8-20288e4bed69 admin admin] The token cache is being invalidate because project 079924db14e4476b8650b08bae2c9dbd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80357) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:44:22.417368 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-ccb4eab5-2d1d-4d4e-a4b8-20288e4bed69 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:22.418233 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-ccb4eab5-2d1d-4d4e-a4b8-20288e4bed69 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:22.419827 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 879/1785] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jun 16 17:44:22 2022] DELETE /identity/v3/projects/079924db14e4476b8650b08bae2c9dbd => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:44:22.425378 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dd2a3c-5739-440b-9d45-1bfec86279a4 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:22.425675 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dd2a3c-5739-440b-9d45-1bfec86279a4 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:22.425956 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dd2a3c-5739-440b-9d45-1bfec86279a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:22.445787 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-93dd2a3c-5739-440b-9d45-1bfec86279a4 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:44:22.447019 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-93dd2a3c-5739-440b-9d45-1bfec86279a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:22.511342 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 907/1786] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:44:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:44:22.519627 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcea682-d57b-4916-a79f-a98ac1009052 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:22.520224 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcea682-d57b-4916-a79f-a98ac1009052 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:22.520224 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcea682-d57b-4916-a79f-a98ac1009052 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:22.539691 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-3fcea682-d57b-4916-a79f-a98ac1009052 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:44:22.540145 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-3fcea682-d57b-4916-a79f-a98ac1009052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:22.549426 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 880/1787] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:44:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:44:22.554219 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dd2a3c-5739-440b-9d45-1bfec86279a4 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:22.564995 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dd2a3c-5739-440b-9d45-1bfec86279a4 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:22.570320 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d96c2ec-b3dc-4852-9e1b-b24652911d85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:22.571549 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d96c2ec-b3dc-4852-9e1b-b24652911d85 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:22.571890 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d96c2ec-b3dc-4852-9e1b-b24652911d85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:22.572407 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d96c2ec-b3dc-4852-9e1b-b24652911d85 admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:22.573033 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d96c2ec-b3dc-4852-9e1b-b24652911d85 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:44:22.573373 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d96c2ec-b3dc-4852-9e1b-b24652911d85 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:22.575166 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d96c2ec-b3dc-4852-9e1b-b24652911d85 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:22.590857 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 908/1788] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jun 16 17:44:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:44:22.595437 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcea682-d57b-4916-a79f-a98ac1009052 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:22.605599 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcea682-d57b-4916-a79f-a98ac1009052 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:22.611582 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ed97c2-b7ab-41f3-b6a3-bd80fc4656d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:22.612345 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ed97c2-b7ab-41f3-b6a3-bd80fc4656d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:22.612737 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ed97c2-b7ab-41f3-b6a3-bd80fc4656d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:22.613306 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ed97c2-b7ab-41f3-b6a3-bd80fc4656d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:22.614668 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ed97c2-b7ab-41f3-b6a3-bd80fc4656d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:22.616247 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ed97c2-b7ab-41f3-b6a3-bd80fc4656d3 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:22.670339 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 881/1789] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jun 16 17:44:22 2022] POST /identity/v3/projects => generated 377 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:44:22.672083 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d96c2ec-b3dc-4852-9e1b-b24652911d85 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:22.682691 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d96c2ec-b3dc-4852-9e1b-b24652911d85 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:22.760154 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94876b34-e58f-4ab4-ae6e-8ce5ec2cb222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:22.761265 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94876b34-e58f-4ab4-ae6e-8ce5ec2cb222 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:22.761935 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94876b34-e58f-4ab4-ae6e-8ce5ec2cb222 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:22.762437 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94876b34-e58f-4ab4-ae6e-8ce5ec2cb222 admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:22.763310 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94876b34-e58f-4ab4-ae6e-8ce5ec2cb222 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:22.766279 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94876b34-e58f-4ab4-ae6e-8ce5ec2cb222 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:22.778422 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:44:22.801357 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 909/1790] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jun 16 17:44:22 2022] POST /identity/v3/users => generated 289 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:44:22.805057 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ed97c2-b7ab-41f3-b6a3-bd80fc4656d3 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:22.818397 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ed97c2-b7ab-41f3-b6a3-bd80fc4656d3 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:22.823393 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c1efbb-57ef-4fbd-8479-67de3106e3e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:22.824142 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c1efbb-57ef-4fbd-8479-67de3106e3e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:22.824701 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c1efbb-57ef-4fbd-8479-67de3106e3e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:22.825046 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c1efbb-57ef-4fbd-8479-67de3106e3e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:22.825939 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c1efbb-57ef-4fbd-8479-67de3106e3e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:22.828649 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c1efbb-57ef-4fbd-8479-67de3106e3e6 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:22.841706 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 882/1791] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:44:22 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:44:22.846395 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94876b34-e58f-4ab4-ae6e-8ce5ec2cb222 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:22.856871 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94876b34-e58f-4ab4-ae6e-8ce5ec2cb222 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:22.870102 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87c4d9e-985d-4d97-851a-40c87965d042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:22.870781 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87c4d9e-985d-4d97-851a-40c87965d042 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:22.871066 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87c4d9e-985d-4d97-851a-40c87965d042 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:22.871628 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87c4d9e-985d-4d97-851a-40c87965d042 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:22.872080 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87c4d9e-985d-4d97-851a-40c87965d042 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:22.873952 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87c4d9e-985d-4d97-851a-40c87965d042 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:22.898867 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 910/1792] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:44:22 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:44:22.899873 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c1efbb-57ef-4fbd-8479-67de3106e3e6 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:22.916149 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c1efbb-57ef-4fbd-8479-67de3106e3e6 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:22.925954 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41944fc-7f30-4901-aec2-79d3adc86c84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:22.928174 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41944fc-7f30-4901-aec2-79d3adc86c84 admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:22.928845 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41944fc-7f30-4901-aec2-79d3adc86c84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:22.929278 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41944fc-7f30-4901-aec2-79d3adc86c84 admin admin] PATH_INFO: `/v3/projects/774476afd17b44b1bae76c13a300577e/users/416a93006e65481c82791c17e04a0fae/roles/0be2075946f54c8485f1c275f746478a` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:22.951145 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41944fc-7f30-4901-aec2-79d3adc86c84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=774476afd17b44b1bae76c13a300577e, user_id=416a93006e65481c82791c17e04a0fae, role_id=0be2075946f54c8485f1c275f746478a)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:22.954757 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41944fc-7f30-4901-aec2-79d3adc86c84 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:22.970992 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 883/1793] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:44:22 2022] PUT /identity/v3/projects/774476afd17b44b1bae76c13a300577e/users/416a93006e65481c82791c17e04a0fae/roles/0be2075946f54c8485f1c275f746478a => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:44:22.982097 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c30370f-0c92-40f6-a3d8-228a89d02731 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:22.982604 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c30370f-0c92-40f6-a3d8-228a89d02731 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:22.982826 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c30370f-0c92-40f6-a3d8-228a89d02731 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:23.019807 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-3c30370f-0c92-40f6-a3d8-228a89d02731 None None] MFA Rules not processed for user `416a93006e65481c82791c17e04a0fae`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:44:23.021605 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-3c30370f-0c92-40f6-a3d8-228a89d02731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:23.130116 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 911/1794] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:44:22 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:44:23.142680 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41944fc-7f30-4901-aec2-79d3adc86c84 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:23.144501 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-b41944fc-7f30-4901-aec2-79d3adc86c84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:23.210698 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41944fc-7f30-4901-aec2-79d3adc86c84 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:23.215004 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b30277-0a1f-41b3-a6a2-6ce380598f1c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd89029c1b3134df1962c041988bead74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:23.215664 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b30277-0a1f-41b3-a6a2-6ce380598f1c service glance] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:23.215952 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b30277-0a1f-41b3-a6a2-6ce380598f1c service glance] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:23.216257 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b30277-0a1f-41b3-a6a2-6ce380598f1c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:23.225478 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b30277-0a1f-41b3-a6a2-6ce380598f1c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:23.226978 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b30277-0a1f-41b3-a6a2-6ce380598f1c service glance] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:23.236919 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 884/1795] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:44:23 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:44:25.013042 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f593123-0877-4bf3-a6c2-35cc1357b870 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:25.013630 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f593123-0877-4bf3-a6c2-35cc1357b870 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:25.026197 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f593123-0877-4bf3-a6c2-35cc1357b870 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:25.066820 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-1f593123-0877-4bf3-a6c2-35cc1357b870 None None] MFA Rules not processed for user `71443c30b4554719b5549493bf3cc00c`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:44:25.066820 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-1f593123-0877-4bf3-a6c2-35cc1357b870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:25.141031 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 912/1796] 10.222.0.42 () {58 vars in 1009 bytes} [Thu Jun 16 17:44:25 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:44:25.155745 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b30277-0a1f-41b3-a6a2-6ce380598f1c service glance] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:25.158190 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-f9b30277-0a1f-41b3-a6a2-6ce380598f1c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:25.268160 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b30277-0a1f-41b3-a6a2-6ce380598f1c service glance] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:25.274231 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf8e45d-0593-4e16-88c5-6422063775a7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47b7a9cd615c41d6983a0007b475038c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:25.274820 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf8e45d-0593-4e16-88c5-6422063775a7 service swift] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:25.275114 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf8e45d-0593-4e16-88c5-6422063775a7 service swift] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:25.275451 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf8e45d-0593-4e16-88c5-6422063775a7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:25.288539 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf8e45d-0593-4e16-88c5-6422063775a7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:25.290150 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf8e45d-0593-4e16-88c5-6422063775a7 service swift] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:25.302558 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 885/1797] 10.222.0.42 () {60 vars in 1384 bytes} [Thu Jun 16 17:44:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 17:44:26.057857 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f593123-0877-4bf3-a6c2-35cc1357b870 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:26.059539 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-1f593123-0877-4bf3-a6c2-35cc1357b870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:26.178853 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f593123-0877-4bf3-a6c2-35cc1357b870 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:26.184682 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b5f807-a97d-4202-9449-3f60dab45809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:26.185874 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b5f807-a97d-4202-9449-3f60dab45809 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:26.186829 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b5f807-a97d-4202-9449-3f60dab45809 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:26.187239 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b5f807-a97d-4202-9449-3f60dab45809 admin admin] PATH_INFO: `/v3/users/a0981ad76c9741f38c62553efcd23c8c` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:26.188966 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b5f807-a97d-4202-9449-3f60dab45809 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0981ad76c9741f38c62553efcd23c8c)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:26.191571 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b5f807-a97d-4202-9449-3f60dab45809 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:26.226034 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:44:26.281966 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-89b5f807-a97d-4202-9449-3f60dab45809 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0981ad76c9741f38c62553efcd23c8c'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:26.297085 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-89b5f807-a97d-4202-9449-3f60dab45809 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0981ad76c9741f38c62553efcd23c8c'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:26.298252 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-89b5f807-a97d-4202-9449-3f60dab45809 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0981ad76c9741f38c62553efcd23c8c'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:26.309878 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-89b5f807-a97d-4202-9449-3f60dab45809 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0981ad76c9741f38c62553efcd23c8c'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:26.327062 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-89b5f807-a97d-4202-9449-3f60dab45809 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0981ad76c9741f38c62553efcd23c8c'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:26.341034 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 913/1798] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jun 16 17:44:26 2022] DELETE /identity/v3/users/a0981ad76c9741f38c62553efcd23c8c => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:44:26.346847 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c484e01a-a4fb-486d-aaf0-18c99e9daa30 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:26.347424 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c484e01a-a4fb-486d-aaf0-18c99e9daa30 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:26.347424 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c484e01a-a4fb-486d-aaf0-18c99e9daa30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:26.353160 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:44:26.378977 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-c484e01a-a4fb-486d-aaf0-18c99e9daa30 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:44:26.380542 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-c484e01a-a4fb-486d-aaf0-18c99e9daa30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:26.456940 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 886/1799] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:44:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:44:26.467661 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b5f807-a97d-4202-9449-3f60dab45809 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:26.469288 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-89b5f807-a97d-4202-9449-3f60dab45809 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:26.601469 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b5f807-a97d-4202-9449-3f60dab45809 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:26.614326 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34a2efe-a667-4ea1-9e32-42bd3009237a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:26.615815 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34a2efe-a667-4ea1-9e32-42bd3009237a service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:26.616330 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34a2efe-a667-4ea1-9e32-42bd3009237a service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:26.616885 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34a2efe-a667-4ea1-9e32-42bd3009237a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:26.630950 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34a2efe-a667-4ea1-9e32-42bd3009237a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:26.632864 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34a2efe-a667-4ea1-9e32-42bd3009237a service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:26.667970 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 914/1800] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:44:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:44:27.204268 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c484e01a-a4fb-486d-aaf0-18c99e9daa30 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:27.205794 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-c484e01a-a4fb-486d-aaf0-18c99e9daa30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:27.221019 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c484e01a-a4fb-486d-aaf0-18c99e9daa30 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:27.226356 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b436dc4d-d481-45d2-a365-a3fb4c0170fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:27.227366 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b436dc4d-d481-45d2-a365-a3fb4c0170fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:27.227761 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b436dc4d-d481-45d2-a365-a3fb4c0170fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:27.228156 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b436dc4d-d481-45d2-a365-a3fb4c0170fc admin admin] PATH_INFO: `/v3/projects/7a38ccb4f65243a9a7253ebe3c4d93fa` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:27.229604 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b436dc4d-d481-45d2-a365-a3fb4c0170fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a38ccb4f65243a9a7253ebe3c4d93fa)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:27.231889 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b436dc4d-d481-45d2-a365-a3fb4c0170fc admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:27.309543 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-b436dc4d-d481-45d2-a365-a3fb4c0170fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a38ccb4f65243a9a7253ebe3c4d93fa', 'request_id': 'req-b436dc4d-d481-45d2-a365-a3fb4c0170fc'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:27.318714 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-b436dc4d-d481-45d2-a365-a3fb4c0170fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a38ccb4f65243a9a7253ebe3c4d93fa', 'request_id': 'req-b436dc4d-d481-45d2-a365-a3fb4c0170fc'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:27.325057 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-b436dc4d-d481-45d2-a365-a3fb4c0170fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a38ccb4f65243a9a7253ebe3c4d93fa', 'request_id': 'req-b436dc4d-d481-45d2-a365-a3fb4c0170fc'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:27.351067 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-b436dc4d-d481-45d2-a365-a3fb4c0170fc admin admin] The token cache is being invalidate because project 7a38ccb4f65243a9a7253ebe3c4d93fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80357) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:44:27.352386 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-b436dc4d-d481-45d2-a365-a3fb4c0170fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:27.353905 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-b436dc4d-d481-45d2-a365-a3fb4c0170fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:27.361416 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 887/1801] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jun 16 17:44:27 2022] DELETE /identity/v3/projects/7a38ccb4f65243a9a7253ebe3c4d93fa => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:44:27.362755 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34a2efe-a667-4ea1-9e32-42bd3009237a service neutron] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:27.374616 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34a2efe-a667-4ea1-9e32-42bd3009237a service neutron] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:27.460039 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225326a8-7078-4a58-8027-2c2425b8a0d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:27.461171 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225326a8-7078-4a58-8027-2c2425b8a0d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:27.461580 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225326a8-7078-4a58-8027-2c2425b8a0d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:27.461970 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225326a8-7078-4a58-8027-2c2425b8a0d5 admin admin] PATH_INFO: `/v3/users/59dfac930e394e368caf5d150cbd6a63` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:27.463310 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225326a8-7078-4a58-8027-2c2425b8a0d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=59dfac930e394e368caf5d150cbd6a63)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:27.465665 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225326a8-7078-4a58-8027-2c2425b8a0d5 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:27.537582 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:44:27.542884 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-225326a8-7078-4a58-8027-2c2425b8a0d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59dfac930e394e368caf5d150cbd6a63'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:27.567585 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-225326a8-7078-4a58-8027-2c2425b8a0d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59dfac930e394e368caf5d150cbd6a63'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:27.568247 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-225326a8-7078-4a58-8027-2c2425b8a0d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59dfac930e394e368caf5d150cbd6a63'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:27.576075 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-225326a8-7078-4a58-8027-2c2425b8a0d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59dfac930e394e368caf5d150cbd6a63'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:27.584944 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-225326a8-7078-4a58-8027-2c2425b8a0d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59dfac930e394e368caf5d150cbd6a63'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:27.592008 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 915/1802] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jun 16 17:44:27 2022] DELETE /identity/v3/users/59dfac930e394e368caf5d150cbd6a63 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:44:28.064537 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b436dc4d-d481-45d2-a365-a3fb4c0170fc admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:28.066548 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-b436dc4d-d481-45d2-a365-a3fb4c0170fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:28.082608 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b436dc4d-d481-45d2-a365-a3fb4c0170fc admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:28.142806 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a57ee09-5f11-4022-acce-3b56334c0170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:28.143753 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a57ee09-5f11-4022-acce-3b56334c0170 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:28.144065 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a57ee09-5f11-4022-acce-3b56334c0170 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:28.144366 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a57ee09-5f11-4022-acce-3b56334c0170 admin admin] PATH_INFO: `/v3/projects/69d9dcc42f874a8db5889ef9143a851d` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:28.145603 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a57ee09-5f11-4022-acce-3b56334c0170 admin admin] RBAC: Authorizing `identity:delete_project(project_id=69d9dcc42f874a8db5889ef9143a851d)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:28.147528 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a57ee09-5f11-4022-acce-3b56334c0170 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:28.192948 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:44:28.214228 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-9a57ee09-5f11-4022-acce-3b56334c0170 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69d9dcc42f874a8db5889ef9143a851d', 'request_id': 'req-9a57ee09-5f11-4022-acce-3b56334c0170'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:28.221693 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-9a57ee09-5f11-4022-acce-3b56334c0170 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69d9dcc42f874a8db5889ef9143a851d', 'request_id': 'req-9a57ee09-5f11-4022-acce-3b56334c0170'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:28.227293 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-9a57ee09-5f11-4022-acce-3b56334c0170 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69d9dcc42f874a8db5889ef9143a851d', 'request_id': 'req-9a57ee09-5f11-4022-acce-3b56334c0170'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:28.239101 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-9a57ee09-5f11-4022-acce-3b56334c0170 admin admin] The token cache is being invalidate because project 69d9dcc42f874a8db5889ef9143a851d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80357) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:44:28.239395 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-9a57ee09-5f11-4022-acce-3b56334c0170 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:28.240109 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-9a57ee09-5f11-4022-acce-3b56334c0170 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:28.241945 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 888/1803] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jun 16 17:44:28 2022] DELETE /identity/v3/projects/69d9dcc42f874a8db5889ef9143a851d => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:44:28.246843 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae655292-c9d9-4787-8215-b4be9ebff536 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:28.247133 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae655292-c9d9-4787-8215-b4be9ebff536 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:28.247437 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae655292-c9d9-4787-8215-b4be9ebff536 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:28.272821 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-ae655292-c9d9-4787-8215-b4be9ebff536 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:44:28.274477 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-ae655292-c9d9-4787-8215-b4be9ebff536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:28.346072 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 916/1804] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:44:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:44:28.354152 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee71717-66f4-44ab-b43f-fc3cb6db3ac4 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:28.354839 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee71717-66f4-44ab-b43f-fc3cb6db3ac4 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:28.354839 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee71717-66f4-44ab-b43f-fc3cb6db3ac4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:28.376972 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-6ee71717-66f4-44ab-b43f-fc3cb6db3ac4 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:44:28.378480 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-6ee71717-66f4-44ab-b43f-fc3cb6db3ac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:28.393543 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae655292-c9d9-4787-8215-b4be9ebff536 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:28.394323 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 889/1805] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:44:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:44:28.411837 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae655292-c9d9-4787-8215-b4be9ebff536 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:28.418698 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfe4fd6-b6d2-432f-93ee-8450849368bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:28.419565 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfe4fd6-b6d2-432f-93ee-8450849368bc admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:28.419992 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfe4fd6-b6d2-432f-93ee-8450849368bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:28.420397 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfe4fd6-b6d2-432f-93ee-8450849368bc admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:28.421206 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfe4fd6-b6d2-432f-93ee-8450849368bc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:44:28.421585 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfe4fd6-b6d2-432f-93ee-8450849368bc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:28.423269 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfe4fd6-b6d2-432f-93ee-8450849368bc admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:28.446563 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee71717-66f4-44ab-b43f-fc3cb6db3ac4 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:28.462267 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 917/1806] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jun 16 17:44:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:44:28.462547 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee71717-66f4-44ab-b43f-fc3cb6db3ac4 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:28.466651 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53196ae1-5590-42b2-973c-4c757c576f73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:28.468299 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53196ae1-5590-42b2-973c-4c757c576f73 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:28.468623 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53196ae1-5590-42b2-973c-4c757c576f73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:28.469084 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53196ae1-5590-42b2-973c-4c757c576f73 admin admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:28.469798 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53196ae1-5590-42b2-973c-4c757c576f73 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:28.471942 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53196ae1-5590-42b2-973c-4c757c576f73 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:28.513594 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 890/1807] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jun 16 17:44:28 2022] POST /identity/v3/projects => generated 367 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:44:28.525195 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfe4fd6-b6d2-432f-93ee-8450849368bc admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:28.541993 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfe4fd6-b6d2-432f-93ee-8450849368bc admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:28.612727 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bc956b-aece-4e6f-8ea9-6cd3512a3e9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:28.613343 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bc956b-aece-4e6f-8ea9-6cd3512a3e9b admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:28.613667 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bc956b-aece-4e6f-8ea9-6cd3512a3e9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:28.613906 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bc956b-aece-4e6f-8ea9-6cd3512a3e9b admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:28.614619 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bc956b-aece-4e6f-8ea9-6cd3512a3e9b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:28.616611 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bc956b-aece-4e6f-8ea9-6cd3512a3e9b admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:28.642681 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 918/1808] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jun 16 17:44:28 2022] POST /identity/v3/users => generated 284 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:44:28.646396 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53196ae1-5590-42b2-973c-4c757c576f73 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:28.665466 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53196ae1-5590-42b2-973c-4c757c576f73 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:28.670423 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e3f27f-ea36-4e2d-a034-91a27f866b7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:28.671094 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e3f27f-ea36-4e2d-a034-91a27f866b7e admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:28.671375 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e3f27f-ea36-4e2d-a034-91a27f866b7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:28.671649 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e3f27f-ea36-4e2d-a034-91a27f866b7e admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:28.672394 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e3f27f-ea36-4e2d-a034-91a27f866b7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:28.673990 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e3f27f-ea36-4e2d-a034-91a27f866b7e admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:28.684686 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 891/1809] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:44:28 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:44:28.689634 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bc956b-aece-4e6f-8ea9-6cd3512a3e9b admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:28.700199 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bc956b-aece-4e6f-8ea9-6cd3512a3e9b admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:28.706050 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c3d6c3-b0a9-4e62-a56e-08b2f28152af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:28.706858 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c3d6c3-b0a9-4e62-a56e-08b2f28152af admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:28.707145 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c3d6c3-b0a9-4e62-a56e-08b2f28152af admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:28.707428 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c3d6c3-b0a9-4e62-a56e-08b2f28152af admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:28.708336 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c3d6c3-b0a9-4e62-a56e-08b2f28152af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:28.710258 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c3d6c3-b0a9-4e62-a56e-08b2f28152af admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:28.724436 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 919/1810] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:44:28 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:44:28.728642 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e3f27f-ea36-4e2d-a034-91a27f866b7e admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:28.745744 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e3f27f-ea36-4e2d-a034-91a27f866b7e admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:28.751345 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd791c5-fa94-45f6-a344-08f201435cb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:28.752266 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd791c5-fa94-45f6-a344-08f201435cb5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:28.752636 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd791c5-fa94-45f6-a344-08f201435cb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:28.752910 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd791c5-fa94-45f6-a344-08f201435cb5 admin admin] PATH_INFO: `/v3/projects/686a45a9bbbf41dd91591d5cb23fff97/users/ad93b7748d1d433280f2fb05286f423a/roles/0be2075946f54c8485f1c275f746478a` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:28.770191 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd791c5-fa94-45f6-a344-08f201435cb5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=686a45a9bbbf41dd91591d5cb23fff97, user_id=ad93b7748d1d433280f2fb05286f423a, role_id=0be2075946f54c8485f1c275f746478a)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:28.772949 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd791c5-fa94-45f6-a344-08f201435cb5 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:28.808310 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 892/1811] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:44:28 2022] PUT /identity/v3/projects/686a45a9bbbf41dd91591d5cb23fff97/users/ad93b7748d1d433280f2fb05286f423a/roles/0be2075946f54c8485f1c275f746478a => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:44:28.820529 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0f3a2d-6c16-4849-bba7-d81182430b76 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:28.820818 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0f3a2d-6c16-4849-bba7-d81182430b76 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:28.821094 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0f3a2d-6c16-4849-bba7-d81182430b76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:28.849639 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-8e0f3a2d-6c16-4849-bba7-d81182430b76 None None] MFA Rules not processed for user `ad93b7748d1d433280f2fb05286f423a`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:44:28.850855 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-8e0f3a2d-6c16-4849-bba7-d81182430b76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:28.893179 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:44:28.953562 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 920/1812] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:44:28 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:44:28.958171 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd791c5-fa94-45f6-a344-08f201435cb5 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:29.085124 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd791c5-fa94-45f6-a344-08f201435cb5 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:29.090515 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3208c526-b165-4a93-a9e6-64bd1828e613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:29.091419 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3208c526-b165-4a93-a9e6-64bd1828e613 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:29.091803 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3208c526-b165-4a93-a9e6-64bd1828e613 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:29.092131 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3208c526-b165-4a93-a9e6-64bd1828e613 admin admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:29.092995 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3208c526-b165-4a93-a9e6-64bd1828e613 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:29.095263 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3208c526-b165-4a93-a9e6-64bd1828e613 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:29.128293 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 893/1813] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jun 16 17:44:28 2022] POST /identity/v3/projects => generated 367 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:44:29.132258 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0f3a2d-6c16-4849-bba7-d81182430b76 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:29.146521 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0f3a2d-6c16-4849-bba7-d81182430b76 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:29.242220 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435db501-c7bb-4c36-bfb6-4fbe4d929f1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:29.243118 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435db501-c7bb-4c36-bfb6-4fbe4d929f1a admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:29.243477 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435db501-c7bb-4c36-bfb6-4fbe4d929f1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:29.244587 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435db501-c7bb-4c36-bfb6-4fbe4d929f1a admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:29.244587 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435db501-c7bb-4c36-bfb6-4fbe4d929f1a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:29.246789 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435db501-c7bb-4c36-bfb6-4fbe4d929f1a admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:29.283658 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 921/1814] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jun 16 17:44:29 2022] POST /identity/v3/users => generated 290 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:44:29.287549 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3208c526-b165-4a93-a9e6-64bd1828e613 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:29.316400 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3208c526-b165-4a93-a9e6-64bd1828e613 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:29.318675 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00b4225-4605-4100-9cc0-c3804a53943f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:29.319398 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00b4225-4605-4100-9cc0-c3804a53943f admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:29.319683 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00b4225-4605-4100-9cc0-c3804a53943f admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:29.319955 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00b4225-4605-4100-9cc0-c3804a53943f admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:29.320754 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00b4225-4605-4100-9cc0-c3804a53943f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:29.322388 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00b4225-4605-4100-9cc0-c3804a53943f admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:29.334405 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 894/1815] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:44:29 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:44:29.338444 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435db501-c7bb-4c36-bfb6-4fbe4d929f1a admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:29.349102 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435db501-c7bb-4c36-bfb6-4fbe4d929f1a admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:29.353736 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a8fad9-4620-4cd4-bd42-4ffecfb4bf03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:29.354618 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a8fad9-4620-4cd4-bd42-4ffecfb4bf03 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:29.354906 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a8fad9-4620-4cd4-bd42-4ffecfb4bf03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:29.355185 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a8fad9-4620-4cd4-bd42-4ffecfb4bf03 admin admin] PATH_INFO: `/v3/projects/46c4d9471e9b4a7fa30d6d2218d3ac42/users/a7856929817447368199f99453cda318/roles/3ca2dcbbd29d4a31a3ab78f2299bc239` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:29.371162 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a8fad9-4620-4cd4-bd42-4ffecfb4bf03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=46c4d9471e9b4a7fa30d6d2218d3ac42, user_id=a7856929817447368199f99453cda318, role_id=3ca2dcbbd29d4a31a3ab78f2299bc239)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:29.374511 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a8fad9-4620-4cd4-bd42-4ffecfb4bf03 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:29.388249 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 922/1816] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:44:29 2022] PUT /identity/v3/projects/46c4d9471e9b4a7fa30d6d2218d3ac42/users/a7856929817447368199f99453cda318/roles/3ca2dcbbd29d4a31a3ab78f2299bc239 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:44:29.392311 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00b4225-4605-4100-9cc0-c3804a53943f admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:29.402509 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00b4225-4605-4100-9cc0-c3804a53943f admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:29.434642 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:44:29.494559 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9857cb03-9876-4cba-b691-efd6baba136b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:29.495069 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9857cb03-9876-4cba-b691-efd6baba136b admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:29.495642 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9857cb03-9876-4cba-b691-efd6baba136b admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:29.496094 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9857cb03-9876-4cba-b691-efd6baba136b admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:29.497107 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9857cb03-9876-4cba-b691-efd6baba136b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:29.499033 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9857cb03-9876-4cba-b691-efd6baba136b admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:29.519219 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 895/1817] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:44:29 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:44:29.521749 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a8fad9-4620-4cd4-bd42-4ffecfb4bf03 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:29.536149 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a8fad9-4620-4cd4-bd42-4ffecfb4bf03 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:29.542713 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75591564-6448-4af9-9fa7-d371ddb16907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:29.544469 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75591564-6448-4af9-9fa7-d371ddb16907 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:29.544469 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75591564-6448-4af9-9fa7-d371ddb16907 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:29.544469 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75591564-6448-4af9-9fa7-d371ddb16907 admin admin] PATH_INFO: `/v3/projects/46c4d9471e9b4a7fa30d6d2218d3ac42/users/a7856929817447368199f99453cda318/roles/3ca2dcbbd29d4a31a3ab78f2299bc239` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:29.547393 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75591564-6448-4af9-9fa7-d371ddb16907 admin admin] RBAC: Authorizing `identity:create_grant(project_id=46c4d9471e9b4a7fa30d6d2218d3ac42, user_id=a7856929817447368199f99453cda318, role_id=3ca2dcbbd29d4a31a3ab78f2299bc239)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:29.551102 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75591564-6448-4af9-9fa7-d371ddb16907 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:29.587146 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 923/1818] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:44:29 2022] PUT /identity/v3/projects/46c4d9471e9b4a7fa30d6d2218d3ac42/users/a7856929817447368199f99453cda318/roles/3ca2dcbbd29d4a31a3ab78f2299bc239 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:44:29.598598 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a8a882-16e5-4965-b7e4-155fafc9c436 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:29.598888 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a8a882-16e5-4965-b7e4-155fafc9c436 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:29.599219 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a8a882-16e5-4965-b7e4-155fafc9c436 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:29.637304 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-f9a8a882-16e5-4965-b7e4-155fafc9c436 None None] MFA Rules not processed for user `a7856929817447368199f99453cda318`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:44:29.639457 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-f9a8a882-16e5-4965-b7e4-155fafc9c436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:29.741898 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 896/1819] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:44:29 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:44:29.754444 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75591564-6448-4af9-9fa7-d371ddb16907 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:29.756340 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-75591564-6448-4af9-9fa7-d371ddb16907 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:29.834457 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75591564-6448-4af9-9fa7-d371ddb16907 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:29.845160 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7beac65f-7f5b-4596-a196-443c1bb97cfe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24d5e2637b7043e08313909d5b06e329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:29.846261 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beac65f-7f5b-4596-a196-443c1bb97cfe service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:29.846429 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beac65f-7f5b-4596-a196-443c1bb97cfe service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:29.846824 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beac65f-7f5b-4596-a196-443c1bb97cfe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:29.872608 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7beac65f-7f5b-4596-a196-443c1bb97cfe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:29.901462 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7beac65f-7f5b-4596-a196-443c1bb97cfe service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:29.918631 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 924/1820] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:44:29 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:44:30.053153 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a8a882-16e5-4965-b7e4-155fafc9c436 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:30.067763 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a8a882-16e5-4965-b7e4-155fafc9c436 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:30.071651 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaf4861-aa0c-4aee-aaa6-2eea2ef03b30 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24d5e2637b7043e08313909d5b06e329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:30.072437 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaf4861-aa0c-4aee-aaa6-2eea2ef03b30 service cinder] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:30.072636 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaf4861-aa0c-4aee-aaa6-2eea2ef03b30 service cinder] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:30.072911 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaf4861-aa0c-4aee-aaa6-2eea2ef03b30 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:30.147550 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaf4861-aa0c-4aee-aaa6-2eea2ef03b30 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:30.149193 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaf4861-aa0c-4aee-aaa6-2eea2ef03b30 service cinder] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:30.164120 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 897/1821] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:44:30 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:44:31.090855 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7beac65f-7f5b-4596-a196-443c1bb97cfe service cinder] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:31.094427 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-7beac65f-7f5b-4596-a196-443c1bb97cfe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:31.144471 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:44:31.220289 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7beac65f-7f5b-4596-a196-443c1bb97cfe service cinder] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:31.226552 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e8aec2-307e-4c86-928b-4ddbcc17a72b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:31.227622 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e8aec2-307e-4c86-928b-4ddbcc17a72b service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:31.228142 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e8aec2-307e-4c86-928b-4ddbcc17a72b service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:31.228993 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e8aec2-307e-4c86-928b-4ddbcc17a72b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:31.231736 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-66e8aec2-307e-4c86-928b-4ddbcc17a72b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:31.251257 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e8aec2-307e-4c86-928b-4ddbcc17a72b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:31.255051 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e8aec2-307e-4c86-928b-4ddbcc17a72b service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:31.293015 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 925/1822] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:44:31 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:44:34.686529 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932753fb-3a65-46c3-8cb1-0b6f2b6edcd7 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:34.687090 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932753fb-3a65-46c3-8cb1-0b6f2b6edcd7 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:34.687390 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932753fb-3a65-46c3-8cb1-0b6f2b6edcd7 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:34.689043 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 898/1823] 10.222.0.42 () {54 vars in 921 bytes} [Thu Jun 16 17:44:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:44:34.730610 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411d5847-d1b9-4a2a-8289-7f9cf2f8c100 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:34.731143 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411d5847-d1b9-4a2a-8289-7f9cf2f8c100 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:34.731370 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411d5847-d1b9-4a2a-8289-7f9cf2f8c100 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:34.761658 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-411d5847-d1b9-4a2a-8289-7f9cf2f8c100 None None] MFA Rules not processed for user `f86c16179a70474a817be9fda33ece9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:44:34.763430 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-411d5847-d1b9-4a2a-8289-7f9cf2f8c100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:34.912598 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 926/1824] 10.222.0.42 () {58 vars in 1018 bytes} [Thu Jun 16 17:44:34 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:44:35.063406 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932753fb-3a65-46c3-8cb1-0b6f2b6edcd7 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:35.066408 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-932753fb-3a65-46c3-8cb1-0b6f2b6edcd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:35.093057 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932753fb-3a65-46c3-8cb1-0b6f2b6edcd7 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:35.101141 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bfb66e-2128-4f43-990f-b61777bcd86e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f86c16179a70474a817be9fda33ece9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:35.102117 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bfb66e-2128-4f43-990f-b61777bcd86e service nova] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:35.102607 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bfb66e-2128-4f43-990f-b61777bcd86e service nova] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:35.103111 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bfb66e-2128-4f43-990f-b61777bcd86e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:35.117750 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bfb66e-2128-4f43-990f-b61777bcd86e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:35.119797 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bfb66e-2128-4f43-990f-b61777bcd86e service nova] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:35.149594 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 899/1825] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:44:35 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:44:35.210083 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d23efe-b019-45da-878b-fda6dc116d50 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:35.210529 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d23efe-b019-45da-878b-fda6dc116d50 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:35.210744 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d23efe-b019-45da-878b-fda6dc116d50 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:35.213040 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 927/1826] 10.222.0.42 () {54 vars in 921 bytes} [Thu Jun 16 17:44:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:44:35.281404 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20951549-165d-45b4-ae1a-a56762d1975e None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:35.281979 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20951549-165d-45b4-ae1a-a56762d1975e None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:35.282259 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20951549-165d-45b4-ae1a-a56762d1975e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:35.319692 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-20951549-165d-45b4-ae1a-a56762d1975e None None] MFA Rules not processed for user `f86c16179a70474a817be9fda33ece9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:44:35.321194 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-20951549-165d-45b4-ae1a-a56762d1975e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:35.335592 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 900/1827] 10.222.0.42 () {58 vars in 1018 bytes} [Thu Jun 16 17:44:35 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:44:35.409768 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d23efe-b019-45da-878b-fda6dc116d50 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:35.448765 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d23efe-b019-45da-878b-fda6dc116d50 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:35.457967 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a542b66-b245-4851-930d-3c306a2af00e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f86c16179a70474a817be9fda33ece9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:35.458780 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a542b66-b245-4851-930d-3c306a2af00e service nova] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:35.459253 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a542b66-b245-4851-930d-3c306a2af00e service nova] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:35.459527 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a542b66-b245-4851-930d-3c306a2af00e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:35.474102 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a542b66-b245-4851-930d-3c306a2af00e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:35.476016 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a542b66-b245-4851-930d-3c306a2af00e service nova] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:35.488910 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 928/1828] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:44:35 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:44:35.759755 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20951549-165d-45b4-ae1a-a56762d1975e None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:35.761343 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-20951549-165d-45b4-ae1a-a56762d1975e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:35.867808 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:44:35.988245 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20951549-165d-45b4-ae1a-a56762d1975e None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:35.994363 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea966e38-39a5-4d72-8820-9a646c53e264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:35.995563 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea966e38-39a5-4d72-8820-9a646c53e264 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:35.996931 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea966e38-39a5-4d72-8820-9a646c53e264 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:35.997826 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea966e38-39a5-4d72-8820-9a646c53e264 admin admin] PATH_INFO: `/v3/users/c2f8205b1ec64438a18b3a91e3076e3f` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:36.000110 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea966e38-39a5-4d72-8820-9a646c53e264 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2f8205b1ec64438a18b3a91e3076e3f)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:36.002277 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea966e38-39a5-4d72-8820-9a646c53e264 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:36.095320 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a542b66-b245-4851-930d-3c306a2af00e service nova] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:36.097542 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-ea966e38-39a5-4d72-8820-9a646c53e264 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2f8205b1ec64438a18b3a91e3076e3f'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:36.098282 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-ea966e38-39a5-4d72-8820-9a646c53e264 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2f8205b1ec64438a18b3a91e3076e3f'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:36.110083 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-ea966e38-39a5-4d72-8820-9a646c53e264 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2f8205b1ec64438a18b3a91e3076e3f'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:36.111738 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a542b66-b245-4851-930d-3c306a2af00e service nova] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:36.114552 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-ea966e38-39a5-4d72-8820-9a646c53e264 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2f8205b1ec64438a18b3a91e3076e3f'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:36.122453 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d898d6b7-384f-44d4-aacb-43ade2762a0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:36.124109 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d898d6b7-384f-44d4-aacb-43ade2762a0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:36.124437 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d898d6b7-384f-44d4-aacb-43ade2762a0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:36.124721 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d898d6b7-384f-44d4-aacb-43ade2762a0d admin admin] PATH_INFO: `/v3/users/ad93b7748d1d433280f2fb05286f423a` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:36.125953 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d898d6b7-384f-44d4-aacb-43ade2762a0d admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad93b7748d1d433280f2fb05286f423a)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:36.132487 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-ea966e38-39a5-4d72-8820-9a646c53e264 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2f8205b1ec64438a18b3a91e3076e3f'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:36.134537 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d898d6b7-384f-44d4-aacb-43ade2762a0d admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:36.153391 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 901/1829] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jun 16 17:44:35 2022] DELETE /identity/v3/users/c2f8205b1ec64438a18b3a91e3076e3f => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:44:36.155479 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99512b15-1241-4f4c-9c80-eed4f6d024e0 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:36.155779 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99512b15-1241-4f4c-9c80-eed4f6d024e0 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:36.156064 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99512b15-1241-4f4c-9c80-eed4f6d024e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:36.179737 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-99512b15-1241-4f4c-9c80-eed4f6d024e0 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:44:36.181318 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-99512b15-1241-4f4c-9c80-eed4f6d024e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:36.217401 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-d898d6b7-384f-44d4-aacb-43ade2762a0d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad93b7748d1d433280f2fb05286f423a'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:36.267438 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-d898d6b7-384f-44d4-aacb-43ade2762a0d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad93b7748d1d433280f2fb05286f423a'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:36.268196 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-d898d6b7-384f-44d4-aacb-43ade2762a0d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad93b7748d1d433280f2fb05286f423a'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:36.279547 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-d898d6b7-384f-44d4-aacb-43ade2762a0d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad93b7748d1d433280f2fb05286f423a'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:36.290541 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-d898d6b7-384f-44d4-aacb-43ade2762a0d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad93b7748d1d433280f2fb05286f423a'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:36.292757 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 902/1830] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:44:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:44:36.300938 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 929/1831] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jun 16 17:44:36 2022] DELETE /identity/v3/users/ad93b7748d1d433280f2fb05286f423a => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:44:36.305381 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f12d439-daa2-44b5-b074-9bbe3395f436 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:36.305635 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d898d6b7-384f-44d4-aacb-43ade2762a0d admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:36.305838 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f12d439-daa2-44b5-b074-9bbe3395f436 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:36.306232 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f12d439-daa2-44b5-b074-9bbe3395f436 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:36.307321 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-d898d6b7-384f-44d4-aacb-43ade2762a0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:36.318079 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:44:36.327448 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-6f12d439-daa2-44b5-b074-9bbe3395f436 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:44:36.329315 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-6f12d439-daa2-44b5-b074-9bbe3395f436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:36.377232 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:44:36.397579 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d898d6b7-384f-44d4-aacb-43ade2762a0d admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:36.403084 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 903/1832] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:44:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:44:36.403766 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63114682-113f-4298-81c2-5137aa74b8ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:36.404497 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63114682-113f-4298-81c2-5137aa74b8ea service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:36.405207 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63114682-113f-4298-81c2-5137aa74b8ea service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:36.405207 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63114682-113f-4298-81c2-5137aa74b8ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:36.413617 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f12d439-daa2-44b5-b074-9bbe3395f436 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:36.419424 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63114682-113f-4298-81c2-5137aa74b8ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:36.421728 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63114682-113f-4298-81c2-5137aa74b8ea service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:36.430196 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f12d439-daa2-44b5-b074-9bbe3395f436 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:36.439076 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 930/1833] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:44:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:44:36.441956 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f4c8bd-0066-4d0f-ad0b-e950215bea5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:36.442963 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f4c8bd-0066-4d0f-ad0b-e950215bea5b service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:36.443425 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f4c8bd-0066-4d0f-ad0b-e950215bea5b service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:36.443894 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f4c8bd-0066-4d0f-ad0b-e950215bea5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:36.457000 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f4c8bd-0066-4d0f-ad0b-e950215bea5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:36.458624 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f4c8bd-0066-4d0f-ad0b-e950215bea5b service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:36.470762 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 904/1834] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:44:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:44:36.878171 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63114682-113f-4298-81c2-5137aa74b8ea service neutron] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:36.879908 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-63114682-113f-4298-81c2-5137aa74b8ea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:36.899019 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63114682-113f-4298-81c2-5137aa74b8ea service neutron] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:36.904644 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9798e690-1ee8-4d87-a11c-cc5bdde7c640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:36.905527 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9798e690-1ee8-4d87-a11c-cc5bdde7c640 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:36.906316 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9798e690-1ee8-4d87-a11c-cc5bdde7c640 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:36.906769 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9798e690-1ee8-4d87-a11c-cc5bdde7c640 admin admin] PATH_INFO: `/v3/projects/686a45a9bbbf41dd91591d5cb23fff97` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:36.908714 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9798e690-1ee8-4d87-a11c-cc5bdde7c640 admin admin] RBAC: Authorizing `identity:delete_project(project_id=686a45a9bbbf41dd91591d5cb23fff97)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:36.910659 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9798e690-1ee8-4d87-a11c-cc5bdde7c640 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:36.962503 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-9798e690-1ee8-4d87-a11c-cc5bdde7c640 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '686a45a9bbbf41dd91591d5cb23fff97', 'request_id': 'req-9798e690-1ee8-4d87-a11c-cc5bdde7c640'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:36.971216 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-9798e690-1ee8-4d87-a11c-cc5bdde7c640 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '686a45a9bbbf41dd91591d5cb23fff97', 'request_id': 'req-9798e690-1ee8-4d87-a11c-cc5bdde7c640'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:36.978779 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-9798e690-1ee8-4d87-a11c-cc5bdde7c640 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '686a45a9bbbf41dd91591d5cb23fff97', 'request_id': 'req-9798e690-1ee8-4d87-a11c-cc5bdde7c640'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:37.000044 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-9798e690-1ee8-4d87-a11c-cc5bdde7c640 admin admin] The token cache is being invalidate because project 686a45a9bbbf41dd91591d5cb23fff97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:44:37.000456 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-9798e690-1ee8-4d87-a11c-cc5bdde7c640 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:37.001139 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-9798e690-1ee8-4d87-a11c-cc5bdde7c640 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:37.004417 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 931/1835] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jun 16 17:44:36 2022] DELETE /identity/v3/projects/686a45a9bbbf41dd91591d5cb23fff97 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:44:37.007078 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f4c8bd-0066-4d0f-ad0b-e950215bea5b service neutron] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:37.017727 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f4c8bd-0066-4d0f-ad0b-e950215bea5b service neutron] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:37.082861 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f76f96-3527-467e-97e1-2e0b7b702b73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:37.083881 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f76f96-3527-467e-97e1-2e0b7b702b73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:37.084177 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f76f96-3527-467e-97e1-2e0b7b702b73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:37.084490 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f76f96-3527-467e-97e1-2e0b7b702b73 admin admin] PATH_INFO: `/v3/users/a7856929817447368199f99453cda318` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:37.085871 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f76f96-3527-467e-97e1-2e0b7b702b73 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7856929817447368199f99453cda318)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:37.088187 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f76f96-3527-467e-97e1-2e0b7b702b73 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:37.158594 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-e8f76f96-3527-467e-97e1-2e0b7b702b73 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7856929817447368199f99453cda318'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:37.159347 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-e8f76f96-3527-467e-97e1-2e0b7b702b73 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7856929817447368199f99453cda318'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:37.168621 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-e8f76f96-3527-467e-97e1-2e0b7b702b73 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7856929817447368199f99453cda318'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:37.169256 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-e8f76f96-3527-467e-97e1-2e0b7b702b73 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7856929817447368199f99453cda318'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:37.188857 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-e8f76f96-3527-467e-97e1-2e0b7b702b73 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7856929817447368199f99453cda318'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:37.207843 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 905/1836] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jun 16 17:44:37 2022] DELETE /identity/v3/users/a7856929817447368199f99453cda318 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:44:37.851137 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9798e690-1ee8-4d87-a11c-cc5bdde7c640 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:37.853097 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-9798e690-1ee8-4d87-a11c-cc5bdde7c640 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:37.867175 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9798e690-1ee8-4d87-a11c-cc5bdde7c640 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:37.990344 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912e810b-25df-49a8-802f-b65e3c2a80ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:37.991381 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912e810b-25df-49a8-802f-b65e3c2a80ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:37.991670 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912e810b-25df-49a8-802f-b65e3c2a80ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:37.991946 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912e810b-25df-49a8-802f-b65e3c2a80ae admin admin] PATH_INFO: `/v3/projects/cb8f47d178b14878b76bb952e5b3571f` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:37.994142 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912e810b-25df-49a8-802f-b65e3c2a80ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb8f47d178b14878b76bb952e5b3571f)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:37.998986 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912e810b-25df-49a8-802f-b65e3c2a80ae admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:38.046331 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:44:38.082254 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-912e810b-25df-49a8-802f-b65e3c2a80ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb8f47d178b14878b76bb952e5b3571f', 'request_id': 'req-912e810b-25df-49a8-802f-b65e3c2a80ae'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:38.093374 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-912e810b-25df-49a8-802f-b65e3c2a80ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb8f47d178b14878b76bb952e5b3571f', 'request_id': 'req-912e810b-25df-49a8-802f-b65e3c2a80ae'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:38.114196 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-912e810b-25df-49a8-802f-b65e3c2a80ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb8f47d178b14878b76bb952e5b3571f', 'request_id': 'req-912e810b-25df-49a8-802f-b65e3c2a80ae'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:38.139298 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-912e810b-25df-49a8-802f-b65e3c2a80ae admin admin] The token cache is being invalidate because project cb8f47d178b14878b76bb952e5b3571f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:44:38.139678 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-912e810b-25df-49a8-802f-b65e3c2a80ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:38.140462 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-912e810b-25df-49a8-802f-b65e3c2a80ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:38.142147 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 932/1837] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jun 16 17:44:37 2022] DELETE /identity/v3/projects/cb8f47d178b14878b76bb952e5b3571f => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:44:38.146227 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f76f96-3527-467e-97e1-2e0b7b702b73 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:38.148470 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-e8f76f96-3527-467e-97e1-2e0b7b702b73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:38.164487 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f76f96-3527-467e-97e1-2e0b7b702b73 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:38.263020 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:44:38.282444 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80f1c86-d7fc-46ac-bc04-fcfc4c1db0b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:38.283746 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80f1c86-d7fc-46ac-bc04-fcfc4c1db0b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:38.286489 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80f1c86-d7fc-46ac-bc04-fcfc4c1db0b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:38.286976 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80f1c86-d7fc-46ac-bc04-fcfc4c1db0b3 admin admin] PATH_INFO: `/v3/users/a3c09123a78442c587b3008078ef4f82` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:38.289296 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80f1c86-d7fc-46ac-bc04-fcfc4c1db0b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3c09123a78442c587b3008078ef4f82)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:38.293489 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80f1c86-d7fc-46ac-bc04-fcfc4c1db0b3 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:38.459398 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-a80f1c86-d7fc-46ac-bc04-fcfc4c1db0b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3c09123a78442c587b3008078ef4f82'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:38.461451 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-a80f1c86-d7fc-46ac-bc04-fcfc4c1db0b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3c09123a78442c587b3008078ef4f82'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:38.488479 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-a80f1c86-d7fc-46ac-bc04-fcfc4c1db0b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3c09123a78442c587b3008078ef4f82'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:38.489464 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-a80f1c86-d7fc-46ac-bc04-fcfc4c1db0b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3c09123a78442c587b3008078ef4f82'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:38.500840 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912e810b-25df-49a8-802f-b65e3c2a80ae admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:38.502303 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-912e810b-25df-49a8-802f-b65e3c2a80ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:38.509981 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-a80f1c86-d7fc-46ac-bc04-fcfc4c1db0b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3c09123a78442c587b3008078ef4f82'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:38.533599 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912e810b-25df-49a8-802f-b65e3c2a80ae admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:38.540905 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 906/1838] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jun 16 17:44:38 2022] DELETE /identity/v3/users/a3c09123a78442c587b3008078ef4f82 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:44:38.555626 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14879069-da37-456b-9d68-7ce667d22256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:38.556744 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14879069-da37-456b-9d68-7ce667d22256 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:38.557192 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14879069-da37-456b-9d68-7ce667d22256 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:38.557621 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14879069-da37-456b-9d68-7ce667d22256 admin admin] PATH_INFO: `/v3/projects/46c4d9471e9b4a7fa30d6d2218d3ac42` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:38.559021 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14879069-da37-456b-9d68-7ce667d22256 admin admin] RBAC: Authorizing `identity:delete_project(project_id=46c4d9471e9b4a7fa30d6d2218d3ac42)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:38.561679 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14879069-da37-456b-9d68-7ce667d22256 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:38.634152 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-14879069-da37-456b-9d68-7ce667d22256 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46c4d9471e9b4a7fa30d6d2218d3ac42', 'request_id': 'req-14879069-da37-456b-9d68-7ce667d22256'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:38.651319 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-14879069-da37-456b-9d68-7ce667d22256 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46c4d9471e9b4a7fa30d6d2218d3ac42', 'request_id': 'req-14879069-da37-456b-9d68-7ce667d22256'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:38.662362 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-14879069-da37-456b-9d68-7ce667d22256 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46c4d9471e9b4a7fa30d6d2218d3ac42', 'request_id': 'req-14879069-da37-456b-9d68-7ce667d22256'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:38.676417 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-14879069-da37-456b-9d68-7ce667d22256 admin admin] The token cache is being invalidate because project 46c4d9471e9b4a7fa30d6d2218d3ac42 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:44:38.676417 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-14879069-da37-456b-9d68-7ce667d22256 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:38.676753 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-14879069-da37-456b-9d68-7ce667d22256 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:38.682267 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 933/1839] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jun 16 17:44:38 2022] DELETE /identity/v3/projects/46c4d9471e9b4a7fa30d6d2218d3ac42 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:44:38.683873 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2953943d-9f5e-4d57-a004-5845ea9bd77c None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:38.684015 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2953943d-9f5e-4d57-a004-5845ea9bd77c None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:38.684232 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2953943d-9f5e-4d57-a004-5845ea9bd77c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:38.710924 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-2953943d-9f5e-4d57-a004-5845ea9bd77c None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:44:38.712261 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-2953943d-9f5e-4d57-a004-5845ea9bd77c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:38.818050 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 907/1840] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:44:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:44:38.827074 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21a92bb-7336-49b7-9d6a-ef9c481bfd23 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:38.827950 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21a92bb-7336-49b7-9d6a-ef9c481bfd23 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:38.828527 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21a92bb-7336-49b7-9d6a-ef9c481bfd23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:38.879400 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-f21a92bb-7336-49b7-9d6a-ef9c481bfd23 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:44:38.881312 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-f21a92bb-7336-49b7-9d6a-ef9c481bfd23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:38.910919 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 934/1841] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:44:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:44:38.914170 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2953943d-9f5e-4d57-a004-5845ea9bd77c None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:38.940319 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2953943d-9f5e-4d57-a004-5845ea9bd77c None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:38.956328 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e78c8f-b873-40a2-8489-a6cc9ebfa202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:38.957134 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e78c8f-b873-40a2-8489-a6cc9ebfa202 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:38.957422 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e78c8f-b873-40a2-8489-a6cc9ebfa202 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:38.957698 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e78c8f-b873-40a2-8489-a6cc9ebfa202 admin admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:38.958529 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e78c8f-b873-40a2-8489-a6cc9ebfa202 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:44:38.958728 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e78c8f-b873-40a2-8489-a6cc9ebfa202 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:38.960348 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e78c8f-b873-40a2-8489-a6cc9ebfa202 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:39.001385 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 908/1842] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jun 16 17:44:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:44:39.002604 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21a92bb-7336-49b7-9d6a-ef9c481bfd23 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:39.004219 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-f21a92bb-7336-49b7-9d6a-ef9c481bfd23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:39.012252 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e78c8f-b873-40a2-8489-a6cc9ebfa202 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:39.025000 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21a92bb-7336-49b7-9d6a-ef9c481bfd23 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:39.032843 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e78c8f-b873-40a2-8489-a6cc9ebfa202 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:39.039139 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec438d5a-e655-4015-ba64-bd0f357ae513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:39.040934 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec438d5a-e655-4015-ba64-bd0f357ae513 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:39.041580 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec438d5a-e655-4015-ba64-bd0f357ae513 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:39.042002 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec438d5a-e655-4015-ba64-bd0f357ae513 admin admin] PATH_INFO: `/v3/projects/7c2d0996d43f43c6a87d3f3c9e427a83` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:39.044153 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec438d5a-e655-4015-ba64-bd0f357ae513 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c2d0996d43f43c6a87d3f3c9e427a83)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:39.048903 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d3ec88-782d-449d-b09c-ce432717635e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:39.050003 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d3ec88-782d-449d-b09c-ce432717635e admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:39.050577 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec438d5a-e655-4015-ba64-bd0f357ae513 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:39.051170 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d3ec88-782d-449d-b09c-ce432717635e admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:39.053488 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d3ec88-782d-449d-b09c-ce432717635e admin admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:39.054424 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d3ec88-782d-449d-b09c-ce432717635e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:39.057659 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d3ec88-782d-449d-b09c-ce432717635e admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:39.131658 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 909/1843] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jun 16 17:44:39 2022] POST /identity/v3/projects => generated 385 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:44:39.136345 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d3ec88-782d-449d-b09c-ce432717635e admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:39.148931 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ec438d5a-e655-4015-ba64-bd0f357ae513 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c2d0996d43f43c6a87d3f3c9e427a83', 'request_id': 'req-ec438d5a-e655-4015-ba64-bd0f357ae513'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:39.164640 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ec438d5a-e655-4015-ba64-bd0f357ae513 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c2d0996d43f43c6a87d3f3c9e427a83', 'request_id': 'req-ec438d5a-e655-4015-ba64-bd0f357ae513'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:39.174773 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ec438d5a-e655-4015-ba64-bd0f357ae513 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c2d0996d43f43c6a87d3f3c9e427a83', 'request_id': 'req-ec438d5a-e655-4015-ba64-bd0f357ae513'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:39.188949 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ec438d5a-e655-4015-ba64-bd0f357ae513 admin admin] The token cache is being invalidate because project 7c2d0996d43f43c6a87d3f3c9e427a83 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:44:39.189240 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ec438d5a-e655-4015-ba64-bd0f357ae513 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:39.190193 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ec438d5a-e655-4015-ba64-bd0f357ae513 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:39.192994 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 935/1844] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jun 16 17:44:39 2022] DELETE /identity/v3/projects/7c2d0996d43f43c6a87d3f3c9e427a83 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:44:39.199506 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d14f1a7-12e6-40d6-a41f-28b45aa789d3 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:39.200036 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d14f1a7-12e6-40d6-a41f-28b45aa789d3 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:39.202988 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d14f1a7-12e6-40d6-a41f-28b45aa789d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:39.229833 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:44:39.233384 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-0d14f1a7-12e6-40d6-a41f-28b45aa789d3 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:44:39.236115 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-0d14f1a7-12e6-40d6-a41f-28b45aa789d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:39.255315 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d3ec88-782d-449d-b09c-ce432717635e admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:39.263185 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=78,keystone:DELETE=19,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:44:39.267264 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9f449d-1e3d-4d21-9486-7e0d033906a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:39.268111 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9f449d-1e3d-4d21-9486-7e0d033906a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:39.268767 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9f449d-1e3d-4d21-9486-7e0d033906a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:39.269115 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9f449d-1e3d-4d21-9486-7e0d033906a3 admin admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:39.270336 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9f449d-1e3d-4d21-9486-7e0d033906a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:39.273399 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9f449d-1e3d-4d21-9486-7e0d033906a3 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:39.309831 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 910/1845] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jun 16 17:44:39 2022] POST /identity/v3/users => generated 293 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:44:39.315643 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9f449d-1e3d-4d21-9486-7e0d033906a3 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:39.322106 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-ea9f449d-1e3d-4d21-9486-7e0d033906a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:39.341908 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9f449d-1e3d-4d21-9486-7e0d033906a3 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:39.347508 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ac914f-e67d-4871-a149-a8d7c54d94c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:39.350401 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ac914f-e67d-4871-a149-a8d7c54d94c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:39.350401 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ac914f-e67d-4871-a149-a8d7c54d94c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:39.350401 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ac914f-e67d-4871-a149-a8d7c54d94c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:39.352680 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ac914f-e67d-4871-a149-a8d7c54d94c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:39.355474 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ac914f-e67d-4871-a149-a8d7c54d94c5 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:39.376724 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 911/1846] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:44:39 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 17:44:39.384527 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ac914f-e67d-4871-a149-a8d7c54d94c5 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:39.386651 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 936/1847] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:44:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:44:39.395020 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5e552b-864b-4c13-9cd4-6b787465ee5e None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:39.395421 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5e552b-864b-4c13-9cd4-6b787465ee5e None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:39.396454 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5e552b-864b-4c13-9cd4-6b787465ee5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:39.409137 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ac914f-e67d-4871-a149-a8d7c54d94c5 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:39.420669 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bad7089-1fbb-4ab0-beba-eb272fedcfd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:39.421875 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bad7089-1fbb-4ab0-beba-eb272fedcfd8 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:39.422338 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bad7089-1fbb-4ab0-beba-eb272fedcfd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:39.422880 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bad7089-1fbb-4ab0-beba-eb272fedcfd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:39.423751 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bad7089-1fbb-4ab0-beba-eb272fedcfd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:39.426523 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bad7089-1fbb-4ab0-beba-eb272fedcfd8 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:39.445504 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-4e5e552b-864b-4c13-9cd4-6b787465ee5e None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:44:39.445504 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-4e5e552b-864b-4c13-9cd4-6b787465ee5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:39.449221 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 912/1848] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:44:39 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 17:44:39.452080 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bad7089-1fbb-4ab0-beba-eb272fedcfd8 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:39.469152 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 937/1849] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:44:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:44:39.469287 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bad7089-1fbb-4ab0-beba-eb272fedcfd8 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:39.470697 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5e552b-864b-4c13-9cd4-6b787465ee5e None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:39.476219 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b005640f-7fb8-4bbe-b771-fe36363fcf9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:39.477522 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b005640f-7fb8-4bbe-b771-fe36363fcf9a admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:39.477958 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b005640f-7fb8-4bbe-b771-fe36363fcf9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:39.478387 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b005640f-7fb8-4bbe-b771-fe36363fcf9a admin admin] PATH_INFO: `/v3/projects/8f016f5f2dea421d8cbc52c838ab1edd/users/5a638453c6f44a51bcb95ad0056f9a6a/roles/0be2075946f54c8485f1c275f746478a` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:39.485129 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5e552b-864b-4c13-9cd4-6b787465ee5e None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:39.501765 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b005640f-7fb8-4bbe-b771-fe36363fcf9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f016f5f2dea421d8cbc52c838ab1edd, user_id=5a638453c6f44a51bcb95ad0056f9a6a, role_id=0be2075946f54c8485f1c275f746478a)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:39.502368 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc3870f-6837-4ace-a69c-15bc5fa88474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:39.503056 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc3870f-6837-4ace-a69c-15bc5fa88474 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:39.503338 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc3870f-6837-4ace-a69c-15bc5fa88474 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:39.503613 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc3870f-6837-4ace-a69c-15bc5fa88474 admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:39.504424 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc3870f-6837-4ace-a69c-15bc5fa88474 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:44:39.504741 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc3870f-6837-4ace-a69c-15bc5fa88474 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:39.505992 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b005640f-7fb8-4bbe-b771-fe36363fcf9a admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:39.506521 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc3870f-6837-4ace-a69c-15bc5fa88474 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:39.539617 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 938/1850] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jun 16 17:44:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:44:39.540593 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc3870f-6837-4ace-a69c-15bc5fa88474 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:39.554252 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 913/1851] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:44:39 2022] PUT /identity/v3/projects/8f016f5f2dea421d8cbc52c838ab1edd/users/5a638453c6f44a51bcb95ad0056f9a6a/roles/0be2075946f54c8485f1c275f746478a => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:44:39.556689 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310835d4-260f-47d8-a26e-78e81093c014 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:39.557020 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310835d4-260f-47d8-a26e-78e81093c014 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:39.557930 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310835d4-260f-47d8-a26e-78e81093c014 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:39.583209 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc3870f-6837-4ace-a69c-15bc5fa88474 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:39.614663 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-310835d4-260f-47d8-a26e-78e81093c014 None None] MFA Rules not processed for user `5a638453c6f44a51bcb95ad0056f9a6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:44:39.616463 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-310835d4-260f-47d8-a26e-78e81093c014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:39.764707 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99f2001-9e13-4d8e-8e7e-186ae5758bdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:39.765834 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99f2001-9e13-4d8e-8e7e-186ae5758bdd admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:39.767844 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99f2001-9e13-4d8e-8e7e-186ae5758bdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:39.769490 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99f2001-9e13-4d8e-8e7e-186ae5758bdd admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:39.772167 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99f2001-9e13-4d8e-8e7e-186ae5758bdd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:39.772816 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 914/1852] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:44:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:44:39.774882 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99f2001-9e13-4d8e-8e7e-186ae5758bdd admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:39.786327 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310835d4-260f-47d8-a26e-78e81093c014 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:39.788494 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-310835d4-260f-47d8-a26e-78e81093c014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:39.861897 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 939/1853] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jun 16 17:44:39 2022] POST /identity/v3/projects => generated 385 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:44:39.870652 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99f2001-9e13-4d8e-8e7e-186ae5758bdd admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:39.921588 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310835d4-260f-47d8-a26e-78e81093c014 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:39.979590 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99f2001-9e13-4d8e-8e7e-186ae5758bdd admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:39.987238 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:44:39.989052 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3889855c-7d67-4c9d-af82-15c4ebad9156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:39.989881 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3889855c-7d67-4c9d-af82-15c4ebad9156 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:39.990164 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3889855c-7d67-4c9d-af82-15c4ebad9156 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:39.990443 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3889855c-7d67-4c9d-af82-15c4ebad9156 admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:39.991239 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3889855c-7d67-4c9d-af82-15c4ebad9156 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:39.993452 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3889855c-7d67-4c9d-af82-15c4ebad9156 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:40.023394 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 940/1854] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jun 16 17:44:39 2022] POST /identity/v3/users => generated 293 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:44:40.035494 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3889855c-7d67-4c9d-af82-15c4ebad9156 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:40.043470 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffed9c6-f409-4e4a-a6a5-d9de35885445 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24d5e2637b7043e08313909d5b06e329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:40.046874 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffed9c6-f409-4e4a-a6a5-d9de35885445 service cinder] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:40.047451 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffed9c6-f409-4e4a-a6a5-d9de35885445 service cinder] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:40.047830 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffed9c6-f409-4e4a-a6a5-d9de35885445 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:40.056850 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3889855c-7d67-4c9d-af82-15c4ebad9156 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:40.071130 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6b64c3-9d82-4f51-a14b-dc9b2b0eae37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:40.072264 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6b64c3-9d82-4f51-a14b-dc9b2b0eae37 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:40.073377 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6b64c3-9d82-4f51-a14b-dc9b2b0eae37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:40.073377 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6b64c3-9d82-4f51-a14b-dc9b2b0eae37 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:40.077988 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6b64c3-9d82-4f51-a14b-dc9b2b0eae37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:40.084202 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6b64c3-9d82-4f51-a14b-dc9b2b0eae37 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:40.107853 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 941/1855] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:44:40 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:44:40.109544 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6b64c3-9d82-4f51-a14b-dc9b2b0eae37 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:40.128077 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:44:40.141505 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6b64c3-9d82-4f51-a14b-dc9b2b0eae37 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:40.151662 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0d8918-c35f-4efc-945c-69a10cb898f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:40.153751 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0d8918-c35f-4efc-945c-69a10cb898f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:40.154159 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0d8918-c35f-4efc-945c-69a10cb898f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:40.154683 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0d8918-c35f-4efc-945c-69a10cb898f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:40.156064 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0d8918-c35f-4efc-945c-69a10cb898f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:40.158918 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0d8918-c35f-4efc-945c-69a10cb898f3 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:40.173665 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffed9c6-f409-4e4a-a6a5-d9de35885445 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:40.175295 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffed9c6-f409-4e4a-a6a5-d9de35885445 service cinder] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:40.189701 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 942/1856] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:44:40 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:44:40.194248 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0d8918-c35f-4efc-945c-69a10cb898f3 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:40.200796 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 915/1857] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:44:39 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 17:44:40.208339 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0d8918-c35f-4efc-945c-69a10cb898f3 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:40.221168 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa65d55f-2a63-42d6-b26b-fd507e882632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:40.223105 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa65d55f-2a63-42d6-b26b-fd507e882632 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:40.223779 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa65d55f-2a63-42d6-b26b-fd507e882632 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:40.224487 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa65d55f-2a63-42d6-b26b-fd507e882632 admin admin] PATH_INFO: `/v3/projects/ffcf9c34ab714b0bac33c6d3c48dad55/users/e82cc06173014a46a5d5102393c525d6/roles/0be2075946f54c8485f1c275f746478a` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:40.262342 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa65d55f-2a63-42d6-b26b-fd507e882632 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffcf9c34ab714b0bac33c6d3c48dad55, user_id=e82cc06173014a46a5d5102393c525d6, role_id=0be2075946f54c8485f1c275f746478a)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:40.266936 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa65d55f-2a63-42d6-b26b-fd507e882632 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:40.287798 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 943/1858] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:44:40 2022] PUT /identity/v3/projects/ffcf9c34ab714b0bac33c6d3c48dad55/users/e82cc06173014a46a5d5102393c525d6/roles/0be2075946f54c8485f1c275f746478a => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:44:40.299274 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11183a7f-704a-4f38-9acb-423d22657962 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:40.299847 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11183a7f-704a-4f38-9acb-423d22657962 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:40.299983 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11183a7f-704a-4f38-9acb-423d22657962 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:40.333375 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-11183a7f-704a-4f38-9acb-423d22657962 None None] MFA Rules not processed for user `e82cc06173014a46a5d5102393c525d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:44:40.335357 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-11183a7f-704a-4f38-9acb-423d22657962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:40.477329 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 916/1859] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:44:40 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:44:40.482090 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa65d55f-2a63-42d6-b26b-fd507e882632 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:40.570467 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa65d55f-2a63-42d6-b26b-fd507e882632 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:40.608431 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fdc66b-981b-4760-815a-d023eec4a5ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:40.611447 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fdc66b-981b-4760-815a-d023eec4a5ed admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:40.611805 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fdc66b-981b-4760-815a-d023eec4a5ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:40.612157 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fdc66b-981b-4760-815a-d023eec4a5ed admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:40.612969 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fdc66b-981b-4760-815a-d023eec4a5ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:40.618702 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fdc66b-981b-4760-815a-d023eec4a5ed admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:40.690486 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 944/1860] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jun 16 17:44:40 2022] POST /identity/v3/projects => generated 387 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:44:40.692078 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11183a7f-704a-4f38-9acb-423d22657962 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:40.707098 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11183a7f-704a-4f38-9acb-423d22657962 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:40.801391 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f7041c-d247-400b-99ca-37846600e8c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:40.802096 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f7041c-d247-400b-99ca-37846600e8c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:40.802382 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f7041c-d247-400b-99ca-37846600e8c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:40.802661 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f7041c-d247-400b-99ca-37846600e8c8 admin admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:40.803360 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f7041c-d247-400b-99ca-37846600e8c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:40.805352 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f7041c-d247-400b-99ca-37846600e8c8 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:40.824879 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:44:40.833079 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 917/1861] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jun 16 17:44:40 2022] POST /identity/v3/users => generated 300 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:44:40.837858 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fdc66b-981b-4760-815a-d023eec4a5ed admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:40.852009 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fdc66b-981b-4760-815a-d023eec4a5ed admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:40.857872 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce795a0-881a-46fa-b940-91b9efb64b19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:40.858831 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce795a0-881a-46fa-b940-91b9efb64b19 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:40.859195 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce795a0-881a-46fa-b940-91b9efb64b19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:40.859551 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce795a0-881a-46fa-b940-91b9efb64b19 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:40.860407 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce795a0-881a-46fa-b940-91b9efb64b19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:40.862095 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce795a0-881a-46fa-b940-91b9efb64b19 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:40.875566 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 945/1862] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:44:40 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:44:40.879809 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f7041c-d247-400b-99ca-37846600e8c8 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:40.895336 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f7041c-d247-400b-99ca-37846600e8c8 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:40.900771 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6c4d1d-5f89-4ecd-b102-dfd4f0e10584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:40.901964 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6c4d1d-5f89-4ecd-b102-dfd4f0e10584 admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:40.902396 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6c4d1d-5f89-4ecd-b102-dfd4f0e10584 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:40.902921 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6c4d1d-5f89-4ecd-b102-dfd4f0e10584 admin admin] PATH_INFO: `/v3/projects/c1adc14dcdc64a29a8a185e030473166/users/e731ecaf83fc4522ac02cb225c07fc75/roles/3ca2dcbbd29d4a31a3ab78f2299bc239` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:40.925862 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6c4d1d-5f89-4ecd-b102-dfd4f0e10584 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1adc14dcdc64a29a8a185e030473166, user_id=e731ecaf83fc4522ac02cb225c07fc75, role_id=3ca2dcbbd29d4a31a3ab78f2299bc239)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:40.929359 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6c4d1d-5f89-4ecd-b102-dfd4f0e10584 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:40.952042 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 918/1863] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:44:40 2022] PUT /identity/v3/projects/c1adc14dcdc64a29a8a185e030473166/users/e731ecaf83fc4522ac02cb225c07fc75/roles/3ca2dcbbd29d4a31a3ab78f2299bc239 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:44:40.955893 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce795a0-881a-46fa-b940-91b9efb64b19 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:41.006716 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce795a0-881a-46fa-b940-91b9efb64b19 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:41.080178 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22f9f68-0ff6-45d5-b986-637e5fa05552 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:41.080627 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22f9f68-0ff6-45d5-b986-637e5fa05552 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:41.080995 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22f9f68-0ff6-45d5-b986-637e5fa05552 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:41.083162 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 919/1864] 10.222.0.42 () {54 vars in 921 bytes} [Thu Jun 16 17:44:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:44:41.099398 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e517b9e5-1772-47e0-bcc9-d094fd7cf4b6 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:41.099926 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e517b9e5-1772-47e0-bcc9-d094fd7cf4b6 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:41.100306 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e517b9e5-1772-47e0-bcc9-d094fd7cf4b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:41.119265 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4774152e-0719-487f-88c2-893d59d6b854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:41.119964 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4774152e-0719-487f-88c2-893d59d6b854 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:41.120250 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4774152e-0719-487f-88c2-893d59d6b854 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:41.120552 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4774152e-0719-487f-88c2-893d59d6b854 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:41.125520 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4774152e-0719-487f-88c2-893d59d6b854 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:41.127349 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-e517b9e5-1772-47e0-bcc9-d094fd7cf4b6 None None] MFA Rules not processed for user `f86c16179a70474a817be9fda33ece9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:44:41.128941 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-e517b9e5-1772-47e0-bcc9-d094fd7cf4b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:41.132438 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4774152e-0719-487f-88c2-893d59d6b854 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:41.182747 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 946/1865] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:44:40 2022] GET /identity/v3/roles => generated 1420 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:44:41.188072 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4774152e-0719-487f-88c2-893d59d6b854 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:41.212629 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4774152e-0719-487f-88c2-893d59d6b854 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:41.219295 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fda2e8-bff5-4042-b3b8-fade824ad0da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:41.220818 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fda2e8-bff5-4042-b3b8-fade824ad0da admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:41.221388 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fda2e8-bff5-4042-b3b8-fade824ad0da admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:41.221857 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fda2e8-bff5-4042-b3b8-fade824ad0da admin admin] PATH_INFO: `/v3/projects/c1adc14dcdc64a29a8a185e030473166/users/e731ecaf83fc4522ac02cb225c07fc75/roles/3ca2dcbbd29d4a31a3ab78f2299bc239` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:41.224288 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fda2e8-bff5-4042-b3b8-fade824ad0da admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1adc14dcdc64a29a8a185e030473166, user_id=e731ecaf83fc4522ac02cb225c07fc75, role_id=3ca2dcbbd29d4a31a3ab78f2299bc239)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:41.228656 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fda2e8-bff5-4042-b3b8-fade824ad0da admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:41.245071 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 947/1866] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:44:41 2022] PUT /identity/v3/projects/c1adc14dcdc64a29a8a185e030473166/users/e731ecaf83fc4522ac02cb225c07fc75/roles/3ca2dcbbd29d4a31a3ab78f2299bc239 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:44:41.257441 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 920/1867] 10.222.0.42 () {58 vars in 1018 bytes} [Thu Jun 16 17:44:41 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:44:41.266162 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d46122-b4f4-4600-819c-e92b43482e0d None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:41.266461 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d46122-b4f4-4600-819c-e92b43482e0d None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:41.266759 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d46122-b4f4-4600-819c-e92b43482e0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:41.305520 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-d2d46122-b4f4-4600-819c-e92b43482e0d None None] MFA Rules not processed for user `e731ecaf83fc4522ac02cb225c07fc75`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:44:41.306425 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-d2d46122-b4f4-4600-819c-e92b43482e0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:41.324531 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:44:41.405339 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 948/1868] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:44:41 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:44:41.420809 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e517b9e5-1772-47e0-bcc9-d094fd7cf4b6 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:41.591425 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e517b9e5-1772-47e0-bcc9-d094fd7cf4b6 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:41.747699 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1620a42-6ff0-4726-8564-e6cba6a1363c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24d5e2637b7043e08313909d5b06e329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:41.751676 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1620a42-6ff0-4726-8564-e6cba6a1363c service cinder] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:41.752252 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1620a42-6ff0-4726-8564-e6cba6a1363c service cinder] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:41.753315 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1620a42-6ff0-4726-8564-e6cba6a1363c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:41.791768 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1620a42-6ff0-4726-8564-e6cba6a1363c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:41.793307 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1620a42-6ff0-4726-8564-e6cba6a1363c service cinder] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:41.819405 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 921/1869] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:44:41 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:44:41.939468 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d46122-b4f4-4600-819c-e92b43482e0d None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:42.038575 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d46122-b4f4-4600-819c-e92b43482e0d None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:42.047405 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a859a79b-c9bf-419c-bbb2-71e546f834c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:42.048328 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a859a79b-c9bf-419c-bbb2-71e546f834c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:42.051530 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a859a79b-c9bf-419c-bbb2-71e546f834c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:42.051830 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a859a79b-c9bf-419c-bbb2-71e546f834c1 admin admin] PATH_INFO: `/v3/users/e82cc06173014a46a5d5102393c525d6` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:42.054276 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a859a79b-c9bf-419c-bbb2-71e546f834c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e82cc06173014a46a5d5102393c525d6)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:42.055668 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a859a79b-c9bf-419c-bbb2-71e546f834c1 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:42.074955 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1620a42-6ff0-4726-8564-e6cba6a1363c service cinder] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:42.077911 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-a1620a42-6ff0-4726-8564-e6cba6a1363c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:42.119852 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:44:42.122923 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-a859a79b-c9bf-419c-bbb2-71e546f834c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e82cc06173014a46a5d5102393c525d6'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:42.142288 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-a859a79b-c9bf-419c-bbb2-71e546f834c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e82cc06173014a46a5d5102393c525d6'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:42.143047 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-a859a79b-c9bf-419c-bbb2-71e546f834c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e82cc06173014a46a5d5102393c525d6'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:42.152825 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-a859a79b-c9bf-419c-bbb2-71e546f834c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e82cc06173014a46a5d5102393c525d6'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:42.164321 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-a859a79b-c9bf-419c-bbb2-71e546f834c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e82cc06173014a46a5d5102393c525d6'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:42.176976 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 949/1870] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jun 16 17:44:41 2022] DELETE /identity/v3/users/e82cc06173014a46a5d5102393c525d6 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:44:42.180737 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78ad00a-5f2e-436a-8e2c-e6aa506f5952 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:42.181558 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78ad00a-5f2e-436a-8e2c-e6aa506f5952 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:42.182478 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78ad00a-5f2e-436a-8e2c-e6aa506f5952 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:42.185018 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1620a42-6ff0-4726-8564-e6cba6a1363c service cinder] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:42.209986 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-d78ad00a-5f2e-436a-8e2c-e6aa506f5952 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:44:42.211461 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-d78ad00a-5f2e-436a-8e2c-e6aa506f5952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:42.247387 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:44:42.288966 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e90994-c93f-45c8-9b4a-c311c9639639 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f86c16179a70474a817be9fda33ece9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:42.290536 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e90994-c93f-45c8-9b4a-c311c9639639 service nova] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:42.291020 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e90994-c93f-45c8-9b4a-c311c9639639 service nova] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:42.291404 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e90994-c93f-45c8-9b4a-c311c9639639 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:42.293763 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-16e90994-c93f-45c8-9b4a-c311c9639639 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:42.302418 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 950/1871] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:44:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:44:42.309161 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e90994-c93f-45c8-9b4a-c311c9639639 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:42.311247 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e90994-c93f-45c8-9b4a-c311c9639639 service nova] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:42.314858 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78ad00a-5f2e-436a-8e2c-e6aa506f5952 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:42.316988 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-d78ad00a-5f2e-436a-8e2c-e6aa506f5952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:42.336404 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 922/1872] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:44:42 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 17:44:42.423185 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78ad00a-5f2e-436a-8e2c-e6aa506f5952 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:42.430255 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305484a2-3d84-4767-a909-6117047020fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:42.431023 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305484a2-3d84-4767-a909-6117047020fe service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:42.431409 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305484a2-3d84-4767-a909-6117047020fe service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:42.431809 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305484a2-3d84-4767-a909-6117047020fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:42.442594 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305484a2-3d84-4767-a909-6117047020fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:42.444033 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305484a2-3d84-4767-a909-6117047020fe service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:42.455442 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 951/1873] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:44:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 17:44:43.020580 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e90994-c93f-45c8-9b4a-c311c9639639 service nova] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:43.022037 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-16e90994-c93f-45c8-9b4a-c311c9639639 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:43.036447 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e90994-c93f-45c8-9b4a-c311c9639639 service nova] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:43.042520 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ec267e-a6c0-44fd-bc73-86316f4356e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:43.053463 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ec267e-a6c0-44fd-bc73-86316f4356e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:43.053760 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ec267e-a6c0-44fd-bc73-86316f4356e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:43.054039 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ec267e-a6c0-44fd-bc73-86316f4356e6 admin admin] PATH_INFO: `/v3/projects/ffcf9c34ab714b0bac33c6d3c48dad55` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:43.061306 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ec267e-a6c0-44fd-bc73-86316f4356e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ffcf9c34ab714b0bac33c6d3c48dad55)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:43.063343 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ec267e-a6c0-44fd-bc73-86316f4356e6 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:43.125135 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-64ec267e-a6c0-44fd-bc73-86316f4356e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ffcf9c34ab714b0bac33c6d3c48dad55', 'request_id': 'req-64ec267e-a6c0-44fd-bc73-86316f4356e6'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:43.136959 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-64ec267e-a6c0-44fd-bc73-86316f4356e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ffcf9c34ab714b0bac33c6d3c48dad55', 'request_id': 'req-64ec267e-a6c0-44fd-bc73-86316f4356e6'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:43.143749 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-64ec267e-a6c0-44fd-bc73-86316f4356e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ffcf9c34ab714b0bac33c6d3c48dad55', 'request_id': 'req-64ec267e-a6c0-44fd-bc73-86316f4356e6'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:43.155850 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-64ec267e-a6c0-44fd-bc73-86316f4356e6 admin admin] The token cache is being invalidate because project ffcf9c34ab714b0bac33c6d3c48dad55 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80357) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:44:43.156137 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-64ec267e-a6c0-44fd-bc73-86316f4356e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:43.156817 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-64ec267e-a6c0-44fd-bc73-86316f4356e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:43.161431 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 923/1874] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jun 16 17:44:43 2022] DELETE /identity/v3/projects/ffcf9c34ab714b0bac33c6d3c48dad55 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:44:43.162403 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305484a2-3d84-4767-a909-6117047020fe service neutron] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:43.172630 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305484a2-3d84-4767-a909-6117047020fe service neutron] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:43.241112 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaaea6e-518c-456a-b8d1-ce7489a225f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:43.242534 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaaea6e-518c-456a-b8d1-ce7489a225f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:43.242818 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaaea6e-518c-456a-b8d1-ce7489a225f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:43.243095 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaaea6e-518c-456a-b8d1-ce7489a225f1 admin admin] PATH_INFO: `/v3/users/e731ecaf83fc4522ac02cb225c07fc75` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:43.244481 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfaaea6e-518c-456a-b8d1-ce7489a225f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e731ecaf83fc4522ac02cb225c07fc75)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:43.246410 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfaaea6e-518c-456a-b8d1-ce7489a225f1 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:43.298036 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-dfaaea6e-518c-456a-b8d1-ce7489a225f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e731ecaf83fc4522ac02cb225c07fc75'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:43.312363 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-dfaaea6e-518c-456a-b8d1-ce7489a225f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e731ecaf83fc4522ac02cb225c07fc75'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:43.313333 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-dfaaea6e-518c-456a-b8d1-ce7489a225f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e731ecaf83fc4522ac02cb225c07fc75'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:43.320541 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-dfaaea6e-518c-456a-b8d1-ce7489a225f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e731ecaf83fc4522ac02cb225c07fc75'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:43.323052 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:44:43.329277 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-dfaaea6e-518c-456a-b8d1-ce7489a225f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e731ecaf83fc4522ac02cb225c07fc75'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:43.336773 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 952/1875] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jun 16 17:44:43 2022] DELETE /identity/v3/users/e731ecaf83fc4522ac02cb225c07fc75 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:44:43.624893 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ec267e-a6c0-44fd-bc73-86316f4356e6 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:43.626450 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-64ec267e-a6c0-44fd-bc73-86316f4356e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:43.642064 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ec267e-a6c0-44fd-bc73-86316f4356e6 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:43.677966 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:44:43.711485 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6855ed5d-e22b-4ab7-a5c2-42466159a929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:43.712366 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6855ed5d-e22b-4ab7-a5c2-42466159a929 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:43.712725 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6855ed5d-e22b-4ab7-a5c2-42466159a929 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:43.713007 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6855ed5d-e22b-4ab7-a5c2-42466159a929 admin admin] PATH_INFO: `/v3/projects/c1adc14dcdc64a29a8a185e030473166` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:43.714237 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6855ed5d-e22b-4ab7-a5c2-42466159a929 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1adc14dcdc64a29a8a185e030473166)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:43.716159 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6855ed5d-e22b-4ab7-a5c2-42466159a929 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:43.771778 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-6855ed5d-e22b-4ab7-a5c2-42466159a929 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1adc14dcdc64a29a8a185e030473166', 'request_id': 'req-6855ed5d-e22b-4ab7-a5c2-42466159a929'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:43.787267 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-6855ed5d-e22b-4ab7-a5c2-42466159a929 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1adc14dcdc64a29a8a185e030473166', 'request_id': 'req-6855ed5d-e22b-4ab7-a5c2-42466159a929'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:43.798253 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-6855ed5d-e22b-4ab7-a5c2-42466159a929 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1adc14dcdc64a29a8a185e030473166', 'request_id': 'req-6855ed5d-e22b-4ab7-a5c2-42466159a929'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:43.812065 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-6855ed5d-e22b-4ab7-a5c2-42466159a929 admin admin] The token cache is being invalidate because project c1adc14dcdc64a29a8a185e030473166 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80357) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:44:43.812428 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-6855ed5d-e22b-4ab7-a5c2-42466159a929 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:43.813158 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-6855ed5d-e22b-4ab7-a5c2-42466159a929 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:43.814804 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 924/1876] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jun 16 17:44:43 2022] DELETE /identity/v3/projects/c1adc14dcdc64a29a8a185e030473166 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:44:43.819563 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9698b23d-a2c2-4278-8758-b5e2e741c59a None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:43.819950 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9698b23d-a2c2-4278-8758-b5e2e741c59a None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:43.820304 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9698b23d-a2c2-4278-8758-b5e2e741c59a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:43.839325 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-9698b23d-a2c2-4278-8758-b5e2e741c59a None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:44:43.840675 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-9698b23d-a2c2-4278-8758-b5e2e741c59a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:43.909704 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 953/1877] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:44:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:44:43.920415 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a142b9-9b08-4538-9928-fd7b0c1b9231 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:43.920750 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a142b9-9b08-4538-9928-fd7b0c1b9231 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:43.921070 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a142b9-9b08-4538-9928-fd7b0c1b9231 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:43.940782 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-10a142b9-9b08-4538-9928-fd7b0c1b9231 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:44:43.941931 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-10a142b9-9b08-4538-9928-fd7b0c1b9231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:43.949817 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 925/1878] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:44:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:44:43.955376 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9698b23d-a2c2-4278-8758-b5e2e741c59a None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:43.968211 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9698b23d-a2c2-4278-8758-b5e2e741c59a None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:43.972973 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f3cc3d-1ffd-4fa9-affe-54b03272fd6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:43.973667 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f3cc3d-1ffd-4fa9-affe-54b03272fd6e admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:43.973953 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f3cc3d-1ffd-4fa9-affe-54b03272fd6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:43.974239 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f3cc3d-1ffd-4fa9-affe-54b03272fd6e admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:43.974977 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f3cc3d-1ffd-4fa9-affe-54b03272fd6e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:44:43.975313 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f3cc3d-1ffd-4fa9-affe-54b03272fd6e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:43.976949 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f3cc3d-1ffd-4fa9-affe-54b03272fd6e admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:43.990434 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 954/1879] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jun 16 17:44:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:44:43.994718 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a142b9-9b08-4538-9928-fd7b0c1b9231 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:44.005265 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a142b9-9b08-4538-9928-fd7b0c1b9231 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:44.010309 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f61f652-d7b2-4ca0-94cf-5648ae67dcbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:44.010964 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f61f652-d7b2-4ca0-94cf-5648ae67dcbd admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:44.011251 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f61f652-d7b2-4ca0-94cf-5648ae67dcbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:44.011530 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f61f652-d7b2-4ca0-94cf-5648ae67dcbd admin admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:44.012228 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f61f652-d7b2-4ca0-94cf-5648ae67dcbd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:44.015484 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f61f652-d7b2-4ca0-94cf-5648ae67dcbd admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:44.052761 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 926/1880] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jun 16 17:44:43 2022] POST /identity/v3/projects => generated 381 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:44:44.054750 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f3cc3d-1ffd-4fa9-affe-54b03272fd6e admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:44.071505 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f3cc3d-1ffd-4fa9-affe-54b03272fd6e admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:44.137133 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d649c00d-66f4-41ed-a632-9dc38358a005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:44.137806 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d649c00d-66f4-41ed-a632-9dc38358a005 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:44.138085 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d649c00d-66f4-41ed-a632-9dc38358a005 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:44.138357 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d649c00d-66f4-41ed-a632-9dc38358a005 admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:44.140067 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d649c00d-66f4-41ed-a632-9dc38358a005 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:44.142013 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d649c00d-66f4-41ed-a632-9dc38358a005 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:44.165313 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 955/1881] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jun 16 17:44:44 2022] POST /identity/v3/users => generated 291 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:44:44.166562 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f61f652-d7b2-4ca0-94cf-5648ae67dcbd admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:44.177615 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f61f652-d7b2-4ca0-94cf-5648ae67dcbd admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:44.182331 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d55822-1aaf-4b46-b0af-a1eb0a722e3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:44.183005 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d55822-1aaf-4b46-b0af-a1eb0a722e3f admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:44.183295 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d55822-1aaf-4b46-b0af-a1eb0a722e3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:44.183576 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d55822-1aaf-4b46-b0af-a1eb0a722e3f admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:44.184312 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d55822-1aaf-4b46-b0af-a1eb0a722e3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:44.186496 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d55822-1aaf-4b46-b0af-a1eb0a722e3f admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:44.200699 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 927/1882] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:44:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 17:44:44.202715 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d649c00d-66f4-41ed-a632-9dc38358a005 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:44.212838 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d649c00d-66f4-41ed-a632-9dc38358a005 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:44.227005 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45c2511-7fbf-41ee-bb32-d941fd31998e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:44.227780 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45c2511-7fbf-41ee-bb32-d941fd31998e admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:44.228330 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45c2511-7fbf-41ee-bb32-d941fd31998e admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:44.228505 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45c2511-7fbf-41ee-bb32-d941fd31998e admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:44.229342 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45c2511-7fbf-41ee-bb32-d941fd31998e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:44.231879 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45c2511-7fbf-41ee-bb32-d941fd31998e admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:44.250805 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 956/1883] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:44:44 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:44:44.251989 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d55822-1aaf-4b46-b0af-a1eb0a722e3f admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:44.262511 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d55822-1aaf-4b46-b0af-a1eb0a722e3f admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:44.268740 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98181d90-305c-41f0-8f0a-3b690d423122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:44.269635 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98181d90-305c-41f0-8f0a-3b690d423122 admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:44.269921 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98181d90-305c-41f0-8f0a-3b690d423122 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:44.270201 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98181d90-305c-41f0-8f0a-3b690d423122 admin admin] PATH_INFO: `/v3/projects/7afcbc1c7b0d4c888e32d9ca925eaf7f/users/83b5c78b81db451abb5e0938fab1574e/roles/0be2075946f54c8485f1c275f746478a` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:44.284900 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98181d90-305c-41f0-8f0a-3b690d423122 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7afcbc1c7b0d4c888e32d9ca925eaf7f, user_id=83b5c78b81db451abb5e0938fab1574e, role_id=0be2075946f54c8485f1c275f746478a)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:44.287544 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98181d90-305c-41f0-8f0a-3b690d423122 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:44.301375 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 928/1884] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:44:44 2022] PUT /identity/v3/projects/7afcbc1c7b0d4c888e32d9ca925eaf7f/users/83b5c78b81db451abb5e0938fab1574e/roles/0be2075946f54c8485f1c275f746478a => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:44:44.313055 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5421521-926b-40ba-9a94-688750898ca2 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:44.313343 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5421521-926b-40ba-9a94-688750898ca2 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:44.313631 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5421521-926b-40ba-9a94-688750898ca2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:44.340892 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-d5421521-926b-40ba-9a94-688750898ca2 None None] MFA Rules not processed for user `83b5c78b81db451abb5e0938fab1574e`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:44:44.342514 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-d5421521-926b-40ba-9a94-688750898ca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:44.390707 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:44:44.411188 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 957/1885] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:44:44 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:44:44.415798 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98181d90-305c-41f0-8f0a-3b690d423122 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:44.476546 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98181d90-305c-41f0-8f0a-3b690d423122 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:44.481199 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ddc420-0f3a-445a-a02e-52fb7f367ad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:44.481892 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ddc420-0f3a-445a-a02e-52fb7f367ad6 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:44.482678 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ddc420-0f3a-445a-a02e-52fb7f367ad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:44.483426 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ddc420-0f3a-445a-a02e-52fb7f367ad6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:44.484141 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ddc420-0f3a-445a-a02e-52fb7f367ad6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:44.486142 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ddc420-0f3a-445a-a02e-52fb7f367ad6 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:44.489443 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:44:44.517136 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 929/1886] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jun 16 17:44:44 2022] POST /identity/v3/projects => generated 381 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:44:44.521178 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5421521-926b-40ba-9a94-688750898ca2 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:44.532323 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5421521-926b-40ba-9a94-688750898ca2 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:44.588092 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dfd2b3-55a5-4419-a764-14e30c7358e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:44.588778 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dfd2b3-55a5-4419-a764-14e30c7358e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:44.589058 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dfd2b3-55a5-4419-a764-14e30c7358e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:44.589332 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dfd2b3-55a5-4419-a764-14e30c7358e5 admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:44.590014 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dfd2b3-55a5-4419-a764-14e30c7358e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:44.592040 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dfd2b3-55a5-4419-a764-14e30c7358e5 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:44.614441 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 958/1887] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jun 16 17:44:44 2022] POST /identity/v3/users => generated 297 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:44:44.618422 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ddc420-0f3a-445a-a02e-52fb7f367ad6 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:44.630213 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ddc420-0f3a-445a-a02e-52fb7f367ad6 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:44.635257 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984906dc-feea-4deb-82c6-2ab8c8e3be13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:44.635964 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984906dc-feea-4deb-82c6-2ab8c8e3be13 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:44.636250 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984906dc-feea-4deb-82c6-2ab8c8e3be13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:44.636735 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984906dc-feea-4deb-82c6-2ab8c8e3be13 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:44.637504 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984906dc-feea-4deb-82c6-2ab8c8e3be13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:44.639235 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984906dc-feea-4deb-82c6-2ab8c8e3be13 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:44.650543 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 930/1888] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:44:44 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:44:44.654973 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dfd2b3-55a5-4419-a764-14e30c7358e5 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:44.664934 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dfd2b3-55a5-4419-a764-14e30c7358e5 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:44.670244 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9e6df1-98da-4dcb-b966-25d1ca9518df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:44.671136 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9e6df1-98da-4dcb-b966-25d1ca9518df admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:44.671420 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9e6df1-98da-4dcb-b966-25d1ca9518df admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:44.671698 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9e6df1-98da-4dcb-b966-25d1ca9518df admin admin] PATH_INFO: `/v3/projects/3d6c05f1c84646f1a632a1c44f45bb19/users/7fa4457e0a7d44d284155c8039c29a23/roles/3ca2dcbbd29d4a31a3ab78f2299bc239` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:44.688479 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9e6df1-98da-4dcb-b966-25d1ca9518df admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d6c05f1c84646f1a632a1c44f45bb19, user_id=7fa4457e0a7d44d284155c8039c29a23, role_id=3ca2dcbbd29d4a31a3ab78f2299bc239)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:44.691162 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9e6df1-98da-4dcb-b966-25d1ca9518df admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:44.704611 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 959/1889] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:44:44 2022] PUT /identity/v3/projects/3d6c05f1c84646f1a632a1c44f45bb19/users/7fa4457e0a7d44d284155c8039c29a23/roles/3ca2dcbbd29d4a31a3ab78f2299bc239 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:44:44.708597 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984906dc-feea-4deb-82c6-2ab8c8e3be13 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:44.719452 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984906dc-feea-4deb-82c6-2ab8c8e3be13 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:44.777626 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f120d1d9-80de-4f7b-afa7-7f634ce49fda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:44.778313 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f120d1d9-80de-4f7b-afa7-7f634ce49fda admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:44.778602 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f120d1d9-80de-4f7b-afa7-7f634ce49fda admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:44.778878 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f120d1d9-80de-4f7b-afa7-7f634ce49fda admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:44.780146 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f120d1d9-80de-4f7b-afa7-7f634ce49fda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:44.781825 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f120d1d9-80de-4f7b-afa7-7f634ce49fda admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:44.793790 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 931/1890] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:44:44 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:44:44.797806 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9e6df1-98da-4dcb-b966-25d1ca9518df admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:44.808134 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9e6df1-98da-4dcb-b966-25d1ca9518df admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:44.815576 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6079fa1-6a4b-466c-9858-93dbb9811faa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:44.816529 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6079fa1-6a4b-466c-9858-93dbb9811faa admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:44.816819 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6079fa1-6a4b-466c-9858-93dbb9811faa admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:44.817102 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6079fa1-6a4b-466c-9858-93dbb9811faa admin admin] PATH_INFO: `/v3/projects/3d6c05f1c84646f1a632a1c44f45bb19/users/7fa4457e0a7d44d284155c8039c29a23/roles/3ca2dcbbd29d4a31a3ab78f2299bc239` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:44.819204 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6079fa1-6a4b-466c-9858-93dbb9811faa admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d6c05f1c84646f1a632a1c44f45bb19, user_id=7fa4457e0a7d44d284155c8039c29a23, role_id=3ca2dcbbd29d4a31a3ab78f2299bc239)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:44.821916 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6079fa1-6a4b-466c-9858-93dbb9811faa admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:44.833979 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 960/1891] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:44:44 2022] PUT /identity/v3/projects/3d6c05f1c84646f1a632a1c44f45bb19/users/7fa4457e0a7d44d284155c8039c29a23/roles/3ca2dcbbd29d4a31a3ab78f2299bc239 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:44:44.844188 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbc1711-f60d-4511-b8f5-3e7a46b33315 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:44.844517 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbc1711-f60d-4511-b8f5-3e7a46b33315 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:44.844801 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbc1711-f60d-4511-b8f5-3e7a46b33315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:44.872156 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-8cbc1711-f60d-4511-b8f5-3e7a46b33315 None None] MFA Rules not processed for user `7fa4457e0a7d44d284155c8039c29a23`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:44:44.873948 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-8cbc1711-f60d-4511-b8f5-3e7a46b33315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:44.956553 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 932/1892] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:44:44 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:44:44.970337 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6079fa1-6a4b-466c-9858-93dbb9811faa admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:44.972117 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-d6079fa1-6a4b-466c-9858-93dbb9811faa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:45.052514 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6079fa1-6a4b-466c-9858-93dbb9811faa admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:45.056136 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608c91f0-195c-4eef-8b5e-174df8fb2b04 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24d5e2637b7043e08313909d5b06e329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:45.056818 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608c91f0-195c-4eef-8b5e-174df8fb2b04 service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:45.057100 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608c91f0-195c-4eef-8b5e-174df8fb2b04 service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:45.057372 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608c91f0-195c-4eef-8b5e-174df8fb2b04 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:45.069681 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608c91f0-195c-4eef-8b5e-174df8fb2b04 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:45.070699 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608c91f0-195c-4eef-8b5e-174df8fb2b04 service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:45.081958 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 961/1893] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:44:44 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:44:45.438975 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbc1711-f60d-4511-b8f5-3e7a46b33315 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:45.448169 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbc1711-f60d-4511-b8f5-3e7a46b33315 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:45.453742 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232e8dac-e960-4bfe-a906-7a48954670aa tempest-VolumeQuotaClassesTest-1315248793 tempest-VolumeQuotaClassesTest-1315248793-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fa4457e0a7d44d284155c8039c29a23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6c05f1c84646f1a632a1c44f45bb19', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:45.454408 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232e8dac-e960-4bfe-a906-7a48954670aa tempest-VolumeQuotaClassesTest-1315248793 tempest-VolumeQuotaClassesTest-1315248793-project-admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:45.454693 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232e8dac-e960-4bfe-a906-7a48954670aa tempest-VolumeQuotaClassesTest-1315248793 tempest-VolumeQuotaClassesTest-1315248793-project-admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:45.454976 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232e8dac-e960-4bfe-a906-7a48954670aa tempest-VolumeQuotaClassesTest-1315248793 tempest-VolumeQuotaClassesTest-1315248793-project-admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:45.455700 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232e8dac-e960-4bfe-a906-7a48954670aa tempest-VolumeQuotaClassesTest-1315248793 tempest-VolumeQuotaClassesTest-1315248793-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:44:45.456005 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232e8dac-e960-4bfe-a906-7a48954670aa tempest-VolumeQuotaClassesTest-1315248793 tempest-VolumeQuotaClassesTest-1315248793-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:45.457646 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232e8dac-e960-4bfe-a906-7a48954670aa tempest-VolumeQuotaClassesTest-1315248793 tempest-VolumeQuotaClassesTest-1315248793-project-admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:45.469975 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 933/1894] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jun 16 17:44:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:44:45.474134 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608c91f0-195c-4eef-8b5e-174df8fb2b04 service cinder] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:45.484658 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608c91f0-195c-4eef-8b5e-174df8fb2b04 service cinder] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:45.490791 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea101c2-d790-43b7-9a7a-b5258aa0b08e tempest-VolumeQuotaClassesTest-1315248793 tempest-VolumeQuotaClassesTest-1315248793-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fa4457e0a7d44d284155c8039c29a23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6c05f1c84646f1a632a1c44f45bb19', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:45.491812 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea101c2-d790-43b7-9a7a-b5258aa0b08e tempest-VolumeQuotaClassesTest-1315248793 tempest-VolumeQuotaClassesTest-1315248793-project-admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:45.492304 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea101c2-d790-43b7-9a7a-b5258aa0b08e tempest-VolumeQuotaClassesTest-1315248793 tempest-VolumeQuotaClassesTest-1315248793-project-admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:45.492762 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea101c2-d790-43b7-9a7a-b5258aa0b08e tempest-VolumeQuotaClassesTest-1315248793 tempest-VolumeQuotaClassesTest-1315248793-project-admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:45.493667 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea101c2-d790-43b7-9a7a-b5258aa0b08e tempest-VolumeQuotaClassesTest-1315248793 tempest-VolumeQuotaClassesTest-1315248793-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:45.502880 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea101c2-d790-43b7-9a7a-b5258aa0b08e tempest-VolumeQuotaClassesTest-1315248793 tempest-VolumeQuotaClassesTest-1315248793-project-admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:45.532530 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 962/1895] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jun 16 17:44:45 2022] POST /identity/v3/projects => generated 354 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:44:45.536502 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232e8dac-e960-4bfe-a906-7a48954670aa tempest-VolumeQuotaClassesTest-1315248793 tempest-VolumeQuotaClassesTest-1315248793-project-admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:45.547053 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232e8dac-e960-4bfe-a906-7a48954670aa tempest-VolumeQuotaClassesTest-1315248793 tempest-VolumeQuotaClassesTest-1315248793-project-admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:45.588241 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:44:45.607974 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ad8e59-a30b-481a-969b-6e1a2671ebb0 tempest-VolumeQuotaClassesTest-1315248793 tempest-VolumeQuotaClassesTest-1315248793-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fa4457e0a7d44d284155c8039c29a23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6c05f1c84646f1a632a1c44f45bb19', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:45.608929 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ad8e59-a30b-481a-969b-6e1a2671ebb0 tempest-VolumeQuotaClassesTest-1315248793 tempest-VolumeQuotaClassesTest-1315248793-project-admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:45.609216 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ad8e59-a30b-481a-969b-6e1a2671ebb0 tempest-VolumeQuotaClassesTest-1315248793 tempest-VolumeQuotaClassesTest-1315248793-project-admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:45.609612 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ad8e59-a30b-481a-969b-6e1a2671ebb0 tempest-VolumeQuotaClassesTest-1315248793 tempest-VolumeQuotaClassesTest-1315248793-project-admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:45.610386 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ad8e59-a30b-481a-969b-6e1a2671ebb0 tempest-VolumeQuotaClassesTest-1315248793 tempest-VolumeQuotaClassesTest-1315248793-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:44:45.610679 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ad8e59-a30b-481a-969b-6e1a2671ebb0 tempest-VolumeQuotaClassesTest-1315248793 tempest-VolumeQuotaClassesTest-1315248793-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:45.612273 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ad8e59-a30b-481a-969b-6e1a2671ebb0 tempest-VolumeQuotaClassesTest-1315248793 tempest-VolumeQuotaClassesTest-1315248793-project-admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:45.624343 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 934/1896] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jun 16 17:44:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:44:45.664713 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea101c2-d790-43b7-9a7a-b5258aa0b08e tempest-VolumeQuotaClassesTest-1315248793 tempest-VolumeQuotaClassesTest-1315248793-project-admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:45.678334 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea101c2-d790-43b7-9a7a-b5258aa0b08e tempest-VolumeQuotaClassesTest-1315248793 tempest-VolumeQuotaClassesTest-1315248793-project-admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:45.683243 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af032381-f935-43fe-bd8c-cacaa52c7bef tempest-VolumeQuotaClassesTest-1315248793 tempest-VolumeQuotaClassesTest-1315248793-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fa4457e0a7d44d284155c8039c29a23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6c05f1c84646f1a632a1c44f45bb19', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:45.684941 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af032381-f935-43fe-bd8c-cacaa52c7bef tempest-VolumeQuotaClassesTest-1315248793 tempest-VolumeQuotaClassesTest-1315248793-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:45.685347 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af032381-f935-43fe-bd8c-cacaa52c7bef tempest-VolumeQuotaClassesTest-1315248793 tempest-VolumeQuotaClassesTest-1315248793-project-admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:45.686339 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af032381-f935-43fe-bd8c-cacaa52c7bef tempest-VolumeQuotaClassesTest-1315248793 tempest-VolumeQuotaClassesTest-1315248793-project-admin] PATH_INFO: `/v3/projects/b1a60081308a429fa0ce1946a5081abb` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:45.687623 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af032381-f935-43fe-bd8c-cacaa52c7bef tempest-VolumeQuotaClassesTest-1315248793 tempest-VolumeQuotaClassesTest-1315248793-project-admin] RBAC: Authorizing `identity:delete_project(project_id=b1a60081308a429fa0ce1946a5081abb)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:45.689578 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af032381-f935-43fe-bd8c-cacaa52c7bef tempest-VolumeQuotaClassesTest-1315248793 tempest-VolumeQuotaClassesTest-1315248793-project-admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:45.709066 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:44:45.741807 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-af032381-f935-43fe-bd8c-cacaa52c7bef tempest-VolumeQuotaClassesTest-1315248793 tempest-VolumeQuotaClassesTest-1315248793-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1a60081308a429fa0ce1946a5081abb', 'request_id': 'req-af032381-f935-43fe-bd8c-cacaa52c7bef'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:45.749187 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-af032381-f935-43fe-bd8c-cacaa52c7bef tempest-VolumeQuotaClassesTest-1315248793 tempest-VolumeQuotaClassesTest-1315248793-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1a60081308a429fa0ce1946a5081abb', 'request_id': 'req-af032381-f935-43fe-bd8c-cacaa52c7bef'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:45.756392 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-af032381-f935-43fe-bd8c-cacaa52c7bef tempest-VolumeQuotaClassesTest-1315248793 tempest-VolumeQuotaClassesTest-1315248793-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1a60081308a429fa0ce1946a5081abb', 'request_id': 'req-af032381-f935-43fe-bd8c-cacaa52c7bef'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:45.767497 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-af032381-f935-43fe-bd8c-cacaa52c7bef tempest-VolumeQuotaClassesTest-1315248793 tempest-VolumeQuotaClassesTest-1315248793-project-admin] The token cache is being invalidate because project b1a60081308a429fa0ce1946a5081abb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:44:45.767785 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-af032381-f935-43fe-bd8c-cacaa52c7bef tempest-VolumeQuotaClassesTest-1315248793 tempest-VolumeQuotaClassesTest-1315248793-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:45.768464 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-af032381-f935-43fe-bd8c-cacaa52c7bef tempest-VolumeQuotaClassesTest-1315248793 tempest-VolumeQuotaClassesTest-1315248793-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:45.770022 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 963/1897] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jun 16 17:44:45 2022] DELETE /identity/v3/projects/b1a60081308a429fa0ce1946a5081abb => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:44:45.956078 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ad8e59-a30b-481a-969b-6e1a2671ebb0 tempest-VolumeQuotaClassesTest-1315248793 tempest-VolumeQuotaClassesTest-1315248793-project-admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:45.957846 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-58ad8e59-a30b-481a-969b-6e1a2671ebb0 tempest-VolumeQuotaClassesTest-1315248793 tempest-VolumeQuotaClassesTest-1315248793-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:46.054733 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ad8e59-a30b-481a-969b-6e1a2671ebb0 tempest-VolumeQuotaClassesTest-1315248793 tempest-VolumeQuotaClassesTest-1315248793-project-admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:46.149112 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a86e475-2b5a-462c-a391-6b7a7e7b3e08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:46.150189 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a86e475-2b5a-462c-a391-6b7a7e7b3e08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:46.150582 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a86e475-2b5a-462c-a391-6b7a7e7b3e08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:46.150941 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a86e475-2b5a-462c-a391-6b7a7e7b3e08 admin admin] PATH_INFO: `/v3/users/83b5c78b81db451abb5e0938fab1574e` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:46.152716 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a86e475-2b5a-462c-a391-6b7a7e7b3e08 admin admin] RBAC: Authorizing `identity:delete_user(user_id=83b5c78b81db451abb5e0938fab1574e)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:46.156678 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a86e475-2b5a-462c-a391-6b7a7e7b3e08 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:46.306004 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-8a86e475-2b5a-462c-a391-6b7a7e7b3e08 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83b5c78b81db451abb5e0938fab1574e'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:46.307256 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-8a86e475-2b5a-462c-a391-6b7a7e7b3e08 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83b5c78b81db451abb5e0938fab1574e'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:46.322420 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-8a86e475-2b5a-462c-a391-6b7a7e7b3e08 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83b5c78b81db451abb5e0938fab1574e'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:46.323702 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-8a86e475-2b5a-462c-a391-6b7a7e7b3e08 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83b5c78b81db451abb5e0938fab1574e'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:46.352487 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-8a86e475-2b5a-462c-a391-6b7a7e7b3e08 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83b5c78b81db451abb5e0938fab1574e'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:46.369988 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 935/1898] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jun 16 17:44:45 2022] DELETE /identity/v3/users/83b5c78b81db451abb5e0938fab1574e => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:44:46.375113 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e0bfb8-a1e8-4248-a825-56a3fc1f7a60 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:46.375604 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e0bfb8-a1e8-4248-a825-56a3fc1f7a60 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:46.376034 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e0bfb8-a1e8-4248-a825-56a3fc1f7a60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:46.412325 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-c8e0bfb8-a1e8-4248-a825-56a3fc1f7a60 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:44:46.416565 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-c8e0bfb8-a1e8-4248-a825-56a3fc1f7a60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:46.571481 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 964/1899] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:44:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:44:46.572862 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a86e475-2b5a-462c-a391-6b7a7e7b3e08 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:46.574894 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-8a86e475-2b5a-462c-a391-6b7a7e7b3e08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:46.582678 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:44:46.709923 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a86e475-2b5a-462c-a391-6b7a7e7b3e08 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:46.734809 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e4dde2-b4c6-4a08-a80b-9273d25907ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:46.739212 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e4dde2-b4c6-4a08-a80b-9273d25907ac service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:46.739573 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e4dde2-b4c6-4a08-a80b-9273d25907ac service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:46.739857 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e4dde2-b4c6-4a08-a80b-9273d25907ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:46.767058 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e4dde2-b4c6-4a08-a80b-9273d25907ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:46.771132 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e4dde2-b4c6-4a08-a80b-9273d25907ac service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:46.805037 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 936/1900] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:44:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:44:47.531487 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e0bfb8-a1e8-4248-a825-56a3fc1f7a60 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:47.532837 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-c8e0bfb8-a1e8-4248-a825-56a3fc1f7a60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:47.550455 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e0bfb8-a1e8-4248-a825-56a3fc1f7a60 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:47.555444 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b70ef9a-6a46-4e0b-a06e-068dfdd82ae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:47.556662 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b70ef9a-6a46-4e0b-a06e-068dfdd82ae6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:47.557179 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b70ef9a-6a46-4e0b-a06e-068dfdd82ae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:47.557628 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b70ef9a-6a46-4e0b-a06e-068dfdd82ae6 admin admin] PATH_INFO: `/v3/projects/7afcbc1c7b0d4c888e32d9ca925eaf7f` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:47.559054 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b70ef9a-6a46-4e0b-a06e-068dfdd82ae6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7afcbc1c7b0d4c888e32d9ca925eaf7f)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:47.561575 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b70ef9a-6a46-4e0b-a06e-068dfdd82ae6 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:47.641500 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-4b70ef9a-6a46-4e0b-a06e-068dfdd82ae6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7afcbc1c7b0d4c888e32d9ca925eaf7f', 'request_id': 'req-4b70ef9a-6a46-4e0b-a06e-068dfdd82ae6'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:47.653758 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-4b70ef9a-6a46-4e0b-a06e-068dfdd82ae6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7afcbc1c7b0d4c888e32d9ca925eaf7f', 'request_id': 'req-4b70ef9a-6a46-4e0b-a06e-068dfdd82ae6'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:47.661511 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-4b70ef9a-6a46-4e0b-a06e-068dfdd82ae6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7afcbc1c7b0d4c888e32d9ca925eaf7f', 'request_id': 'req-4b70ef9a-6a46-4e0b-a06e-068dfdd82ae6'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:47.673813 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-4b70ef9a-6a46-4e0b-a06e-068dfdd82ae6 admin admin] The token cache is being invalidate because project 7afcbc1c7b0d4c888e32d9ca925eaf7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:44:47.674113 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-4b70ef9a-6a46-4e0b-a06e-068dfdd82ae6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:47.674907 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-4b70ef9a-6a46-4e0b-a06e-068dfdd82ae6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:47.680618 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 965/1901] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jun 16 17:44:47 2022] DELETE /identity/v3/projects/7afcbc1c7b0d4c888e32d9ca925eaf7f => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:44:47.682838 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e4dde2-b4c6-4a08-a80b-9273d25907ac service neutron] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:47.694661 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e4dde2-b4c6-4a08-a80b-9273d25907ac service neutron] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:47.795694 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54136238-abbb-4981-af6a-12b423d494c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:47.797319 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54136238-abbb-4981-af6a-12b423d494c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:47.798161 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54136238-abbb-4981-af6a-12b423d494c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:47.798739 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54136238-abbb-4981-af6a-12b423d494c2 admin admin] PATH_INFO: `/v3/users/7fa4457e0a7d44d284155c8039c29a23` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:47.800861 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54136238-abbb-4981-af6a-12b423d494c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7fa4457e0a7d44d284155c8039c29a23)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:47.803473 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54136238-abbb-4981-af6a-12b423d494c2 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:47.902019 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-54136238-abbb-4981-af6a-12b423d494c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7fa4457e0a7d44d284155c8039c29a23'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:47.902964 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-54136238-abbb-4981-af6a-12b423d494c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7fa4457e0a7d44d284155c8039c29a23'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:47.916918 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-54136238-abbb-4981-af6a-12b423d494c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7fa4457e0a7d44d284155c8039c29a23'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:47.917556 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-54136238-abbb-4981-af6a-12b423d494c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7fa4457e0a7d44d284155c8039c29a23'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:47.932000 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-54136238-abbb-4981-af6a-12b423d494c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7fa4457e0a7d44d284155c8039c29a23'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:47.945348 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 937/1902] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jun 16 17:44:47 2022] DELETE /identity/v3/users/7fa4457e0a7d44d284155c8039c29a23 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:44:48.258752 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b70ef9a-6a46-4e0b-a06e-068dfdd82ae6 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:48.260199 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-4b70ef9a-6a46-4e0b-a06e-068dfdd82ae6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:48.275737 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b70ef9a-6a46-4e0b-a06e-068dfdd82ae6 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:48.316817 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:44:48.358300 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb50f94-0e65-4bd3-9fb4-8e4192523988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:48.359687 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb50f94-0e65-4bd3-9fb4-8e4192523988 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:48.359982 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb50f94-0e65-4bd3-9fb4-8e4192523988 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:48.360258 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb50f94-0e65-4bd3-9fb4-8e4192523988 admin admin] PATH_INFO: `/v3/projects/3d6c05f1c84646f1a632a1c44f45bb19` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:48.361808 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb50f94-0e65-4bd3-9fb4-8e4192523988 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d6c05f1c84646f1a632a1c44f45bb19)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:48.363832 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb50f94-0e65-4bd3-9fb4-8e4192523988 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:48.427335 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-7fb50f94-0e65-4bd3-9fb4-8e4192523988 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d6c05f1c84646f1a632a1c44f45bb19', 'request_id': 'req-7fb50f94-0e65-4bd3-9fb4-8e4192523988'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:48.440030 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-7fb50f94-0e65-4bd3-9fb4-8e4192523988 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d6c05f1c84646f1a632a1c44f45bb19', 'request_id': 'req-7fb50f94-0e65-4bd3-9fb4-8e4192523988'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:48.446688 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-7fb50f94-0e65-4bd3-9fb4-8e4192523988 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d6c05f1c84646f1a632a1c44f45bb19', 'request_id': 'req-7fb50f94-0e65-4bd3-9fb4-8e4192523988'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:48.457119 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-7fb50f94-0e65-4bd3-9fb4-8e4192523988 admin admin] The token cache is being invalidate because project 3d6c05f1c84646f1a632a1c44f45bb19 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:44:48.457431 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-7fb50f94-0e65-4bd3-9fb4-8e4192523988 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:48.458052 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-7fb50f94-0e65-4bd3-9fb4-8e4192523988 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:48.459604 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 966/1903] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jun 16 17:44:48 2022] DELETE /identity/v3/projects/3d6c05f1c84646f1a632a1c44f45bb19 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:44:48.464457 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7f420e-9b8d-4b0d-a0f5-58acad3cc29c None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:48.464762 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7f420e-9b8d-4b0d-a0f5-58acad3cc29c None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:48.465046 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7f420e-9b8d-4b0d-a0f5-58acad3cc29c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:48.485293 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-bb7f420e-9b8d-4b0d-a0f5-58acad3cc29c None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:44:48.486544 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-bb7f420e-9b8d-4b0d-a0f5-58acad3cc29c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:48.520500 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:44:48.589298 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 938/1904] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:44:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:44:48.593899 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57357d0-cfba-4979-b6a2-fc693d0512d0 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:48.594190 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57357d0-cfba-4979-b6a2-fc693d0512d0 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:48.594539 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57357d0-cfba-4979-b6a2-fc693d0512d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:48.615886 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-c57357d0-cfba-4979-b6a2-fc693d0512d0 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:44:48.617292 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-c57357d0-cfba-4979-b6a2-fc693d0512d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:48.631338 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7f420e-9b8d-4b0d-a0f5-58acad3cc29c None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:48.632459 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 967/1905] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:44:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:44:48.642639 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7f420e-9b8d-4b0d-a0f5-58acad3cc29c None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:48.647631 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b2755c-b6eb-4ca4-977d-23bb08c5d6ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:48.648309 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b2755c-b6eb-4ca4-977d-23bb08c5d6ec admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:48.648658 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b2755c-b6eb-4ca4-977d-23bb08c5d6ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:48.648934 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b2755c-b6eb-4ca4-977d-23bb08c5d6ec admin admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:48.649670 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b2755c-b6eb-4ca4-977d-23bb08c5d6ec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:44:48.649978 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b2755c-b6eb-4ca4-977d-23bb08c5d6ec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:48.651735 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b2755c-b6eb-4ca4-977d-23bb08c5d6ec admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:48.664458 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 939/1906] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jun 16 17:44:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:44:48.668763 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57357d0-cfba-4979-b6a2-fc693d0512d0 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:48.679277 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57357d0-cfba-4979-b6a2-fc693d0512d0 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:48.684319 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75165d1-e8cc-4319-b873-dda7aa23aab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:48.685017 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75165d1-e8cc-4319-b873-dda7aa23aab2 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:48.685361 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75165d1-e8cc-4319-b873-dda7aa23aab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:48.685643 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75165d1-e8cc-4319-b873-dda7aa23aab2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:48.686405 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75165d1-e8cc-4319-b873-dda7aa23aab2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:48.688441 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75165d1-e8cc-4319-b873-dda7aa23aab2 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:48.720672 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 968/1907] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jun 16 17:44:48 2022] POST /identity/v3/projects => generated 393 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:44:48.721903 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b2755c-b6eb-4ca4-977d-23bb08c5d6ec admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:48.733495 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b2755c-b6eb-4ca4-977d-23bb08c5d6ec admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:48.799159 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed09915-d963-4889-b2c9-6c0d35bbed82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:48.799839 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed09915-d963-4889-b2c9-6c0d35bbed82 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:48.800126 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed09915-d963-4889-b2c9-6c0d35bbed82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:48.800441 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed09915-d963-4889-b2c9-6c0d35bbed82 admin admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:48.801137 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed09915-d963-4889-b2c9-6c0d35bbed82 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:48.803059 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed09915-d963-4889-b2c9-6c0d35bbed82 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:48.822592 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 940/1908] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jun 16 17:44:48 2022] POST /identity/v3/users => generated 297 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:44:48.827332 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75165d1-e8cc-4319-b873-dda7aa23aab2 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:48.837649 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75165d1-e8cc-4319-b873-dda7aa23aab2 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:48.843088 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccb4e29-cbea-4eb9-b803-f8e93493b45b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:48.843749 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccb4e29-cbea-4eb9-b803-f8e93493b45b admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:48.844036 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccb4e29-cbea-4eb9-b803-f8e93493b45b admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:48.844317 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccb4e29-cbea-4eb9-b803-f8e93493b45b admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:48.845072 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccb4e29-cbea-4eb9-b803-f8e93493b45b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:48.846672 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccb4e29-cbea-4eb9-b803-f8e93493b45b admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:48.857620 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 969/1909] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:44:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:44:48.861588 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed09915-d963-4889-b2c9-6c0d35bbed82 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:48.872478 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed09915-d963-4889-b2c9-6c0d35bbed82 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:48.877724 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2f4f1a-b1f9-441d-a979-0fc755f2ecd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:48.878530 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2f4f1a-b1f9-441d-a979-0fc755f2ecd4 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:48.878819 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2f4f1a-b1f9-441d-a979-0fc755f2ecd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:48.879096 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2f4f1a-b1f9-441d-a979-0fc755f2ecd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:48.879918 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2f4f1a-b1f9-441d-a979-0fc755f2ecd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:48.884454 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2f4f1a-b1f9-441d-a979-0fc755f2ecd4 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:48.895306 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 941/1910] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:44:48 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:44:48.899703 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccb4e29-cbea-4eb9-b803-f8e93493b45b admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:48.911862 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccb4e29-cbea-4eb9-b803-f8e93493b45b admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:48.916666 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46afa2c0-594d-4ddb-86ed-5d56c7af77b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:48.917554 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46afa2c0-594d-4ddb-86ed-5d56c7af77b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:48.917837 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46afa2c0-594d-4ddb-86ed-5d56c7af77b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:48.918117 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46afa2c0-594d-4ddb-86ed-5d56c7af77b1 admin admin] PATH_INFO: `/v3/projects/31e96b2099aa491c977c03f25f8bfd83/users/8d53e94e50f54fd19ad53e8ac048a040/roles/0be2075946f54c8485f1c275f746478a` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:48.944549 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46afa2c0-594d-4ddb-86ed-5d56c7af77b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31e96b2099aa491c977c03f25f8bfd83, user_id=8d53e94e50f54fd19ad53e8ac048a040, role_id=0be2075946f54c8485f1c275f746478a)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:48.947661 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46afa2c0-594d-4ddb-86ed-5d56c7af77b1 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:48.963028 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 970/1911] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:44:48 2022] PUT /identity/v3/projects/31e96b2099aa491c977c03f25f8bfd83/users/8d53e94e50f54fd19ad53e8ac048a040/roles/0be2075946f54c8485f1c275f746478a => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:44:48.975018 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb373de-e0f4-4502-bc86-c692ffc853e4 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:48.975400 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb373de-e0f4-4502-bc86-c692ffc853e4 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:48.975780 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb373de-e0f4-4502-bc86-c692ffc853e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:49.006040 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-1fb373de-e0f4-4502-bc86-c692ffc853e4 None None] MFA Rules not processed for user `8d53e94e50f54fd19ad53e8ac048a040`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:44:49.015734 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-1fb373de-e0f4-4502-bc86-c692ffc853e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:49.096260 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 942/1912] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:44:48 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:44:49.100745 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46afa2c0-594d-4ddb-86ed-5d56c7af77b1 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:49.212778 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:44:49.216994 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46afa2c0-594d-4ddb-86ed-5d56c7af77b1 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:49.223362 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b346151-1fca-4bd4-a362-37feb7a6b1a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:49.224244 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b346151-1fca-4bd4-a362-37feb7a6b1a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:49.224629 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b346151-1fca-4bd4-a362-37feb7a6b1a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:49.224978 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b346151-1fca-4bd4-a362-37feb7a6b1a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:49.225923 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b346151-1fca-4bd4-a362-37feb7a6b1a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:49.228169 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b346151-1fca-4bd4-a362-37feb7a6b1a9 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:49.261017 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 971/1913] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jun 16 17:44:49 2022] POST /identity/v3/projects => generated 393 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:44:49.265990 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb373de-e0f4-4502-bc86-c692ffc853e4 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:49.278150 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb373de-e0f4-4502-bc86-c692ffc853e4 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:49.306203 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:44:49.352851 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab744a2-9c28-4fec-a086-1de717781e68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:49.352851 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab744a2-9c28-4fec-a086-1de717781e68 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:49.353039 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab744a2-9c28-4fec-a086-1de717781e68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:49.353257 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab744a2-9c28-4fec-a086-1de717781e68 admin admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:49.353785 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b346151-1fca-4bd4-a362-37feb7a6b1a9 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:49.353896 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab744a2-9c28-4fec-a086-1de717781e68 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:49.355849 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab744a2-9c28-4fec-a086-1de717781e68 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:49.355966 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-1b346151-1fca-4bd4-a362-37feb7a6b1a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:49.370015 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b346151-1fca-4bd4-a362-37feb7a6b1a9 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:49.380654 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 943/1914] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jun 16 17:44:49 2022] POST /identity/v3/users => generated 303 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:44:49.381806 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f70f428-83a5-4265-a4a2-32b8d65e1556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:49.382688 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f70f428-83a5-4265-a4a2-32b8d65e1556 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:49.382964 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f70f428-83a5-4265-a4a2-32b8d65e1556 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:49.383494 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f70f428-83a5-4265-a4a2-32b8d65e1556 admin admin] PATH_INFO: `/v3/users/5a638453c6f44a51bcb95ad0056f9a6a` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:49.384888 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab744a2-9c28-4fec-a086-1de717781e68 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:49.385838 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f70f428-83a5-4265-a4a2-32b8d65e1556 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a638453c6f44a51bcb95ad0056f9a6a)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:49.387757 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f70f428-83a5-4265-a4a2-32b8d65e1556 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:49.398990 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab744a2-9c28-4fec-a086-1de717781e68 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:49.408540 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e271901-e7e7-4fed-ab85-729417f7b0f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:49.408540 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e271901-e7e7-4fed-ab85-729417f7b0f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:49.408540 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e271901-e7e7-4fed-ab85-729417f7b0f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:49.409071 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e271901-e7e7-4fed-ab85-729417f7b0f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:49.409659 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e271901-e7e7-4fed-ab85-729417f7b0f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:49.411549 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e271901-e7e7-4fed-ab85-729417f7b0f9 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:49.468008 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 944/1915] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:44:49 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:44:49.469763 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e271901-e7e7-4fed-ab85-729417f7b0f9 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:49.482603 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e271901-e7e7-4fed-ab85-729417f7b0f9 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:49.489380 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-3f70f428-83a5-4265-a4a2-32b8d65e1556 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a638453c6f44a51bcb95ad0056f9a6a'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:49.495168 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cd3da0-b23f-4961-8105-16af960be434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:49.496074 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cd3da0-b23f-4961-8105-16af960be434 admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:49.496567 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cd3da0-b23f-4961-8105-16af960be434 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:49.496854 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cd3da0-b23f-4961-8105-16af960be434 admin admin] PATH_INFO: `/v3/projects/9b4f4c4df0f94237b1c4f1cfc5b9b53a/users/423f9ecab4ff4713b607d0aa94aca899/roles/3ca2dcbbd29d4a31a3ab78f2299bc239` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:49.503546 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-3f70f428-83a5-4265-a4a2-32b8d65e1556 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a638453c6f44a51bcb95ad0056f9a6a'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:49.504360 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-3f70f428-83a5-4265-a4a2-32b8d65e1556 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a638453c6f44a51bcb95ad0056f9a6a'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:49.511829 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-3f70f428-83a5-4265-a4a2-32b8d65e1556 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a638453c6f44a51bcb95ad0056f9a6a'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:49.516768 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cd3da0-b23f-4961-8105-16af960be434 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b4f4c4df0f94237b1c4f1cfc5b9b53a, user_id=423f9ecab4ff4713b607d0aa94aca899, role_id=3ca2dcbbd29d4a31a3ab78f2299bc239)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:49.519604 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cd3da0-b23f-4961-8105-16af960be434 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:49.521936 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-3f70f428-83a5-4265-a4a2-32b8d65e1556 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a638453c6f44a51bcb95ad0056f9a6a'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:49.532420 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 972/1916] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jun 16 17:44:49 2022] DELETE /identity/v3/users/5a638453c6f44a51bcb95ad0056f9a6a => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:44:49.538426 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bd33d1-61b4-4249-baa9-ddc6492341dc None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:49.538689 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bd33d1-61b4-4249-baa9-ddc6492341dc None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:49.539368 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bd33d1-61b4-4249-baa9-ddc6492341dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:49.544436 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 945/1917] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:44:49 2022] PUT /identity/v3/projects/9b4f4c4df0f94237b1c4f1cfc5b9b53a/users/423f9ecab4ff4713b607d0aa94aca899/roles/3ca2dcbbd29d4a31a3ab78f2299bc239 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:44:49.547320 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cd3da0-b23f-4961-8105-16af960be434 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:49.553449 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-24cd3da0-b23f-4961-8105-16af960be434 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:49.560882 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-35bd33d1-61b4-4249-baa9-ddc6492341dc None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:44:49.562283 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-35bd33d1-61b4-4249-baa9-ddc6492341dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:49.628748 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 973/1918] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:44:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:44:49.635678 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cd3da0-b23f-4961-8105-16af960be434 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:49.645394 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad2c73f-950f-4fa3-9614-75c14367f85c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:49.646608 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad2c73f-950f-4fa3-9614-75c14367f85c admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:49.646932 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad2c73f-950f-4fa3-9614-75c14367f85c admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:49.647243 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad2c73f-950f-4fa3-9614-75c14367f85c admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:49.647995 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad2c73f-950f-4fa3-9614-75c14367f85c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:49.648462 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bd33d1-61b4-4249-baa9-ddc6492341dc None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:49.649674 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad2c73f-950f-4fa3-9614-75c14367f85c admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:49.649854 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-35bd33d1-61b4-4249-baa9-ddc6492341dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:49.661271 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 946/1919] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:44:49 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 17:44:49.665131 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad2c73f-950f-4fa3-9614-75c14367f85c admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:49.676013 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad2c73f-950f-4fa3-9614-75c14367f85c admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:49.681373 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbe6e0f-ae06-4db6-b692-f6f98f94d8a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:49.682266 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbe6e0f-ae06-4db6-b692-f6f98f94d8a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:49.682551 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbe6e0f-ae06-4db6-b692-f6f98f94d8a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:49.682827 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbe6e0f-ae06-4db6-b692-f6f98f94d8a9 admin admin] PATH_INFO: `/v3/projects/9b4f4c4df0f94237b1c4f1cfc5b9b53a/users/423f9ecab4ff4713b607d0aa94aca899/roles/3ca2dcbbd29d4a31a3ab78f2299bc239` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:49.685352 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbe6e0f-ae06-4db6-b692-f6f98f94d8a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b4f4c4df0f94237b1c4f1cfc5b9b53a, user_id=423f9ecab4ff4713b607d0aa94aca899, role_id=3ca2dcbbd29d4a31a3ab78f2299bc239)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:49.687551 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbe6e0f-ae06-4db6-b692-f6f98f94d8a9 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:49.699971 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 947/1920] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:44:49 2022] PUT /identity/v3/projects/9b4f4c4df0f94237b1c4f1cfc5b9b53a/users/423f9ecab4ff4713b607d0aa94aca899/roles/3ca2dcbbd29d4a31a3ab78f2299bc239 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:44:49.710804 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaddf6be-dc95-4a90-a9ab-8c8354d71d01 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:49.711092 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaddf6be-dc95-4a90-a9ab-8c8354d71d01 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:49.711369 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaddf6be-dc95-4a90-a9ab-8c8354d71d01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:49.713970 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bd33d1-61b4-4249-baa9-ddc6492341dc None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:49.720698 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:44:49.743803 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-aaddf6be-dc95-4a90-a9ab-8c8354d71d01 None None] MFA Rules not processed for user `423f9ecab4ff4713b607d0aa94aca899`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:44:49.745950 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-aaddf6be-dc95-4a90-a9ab-8c8354d71d01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:49.819408 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9879819-3c50-4de7-9fdc-7208f25a65ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:49.820221 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9879819-3c50-4de7-9fdc-7208f25a65ae service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:49.821582 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9879819-3c50-4de7-9fdc-7208f25a65ae service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:49.821974 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9879819-3c50-4de7-9fdc-7208f25a65ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:49.827703 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:44:49.883458 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 948/1921] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:44:49 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:44:49.893166 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaddf6be-dc95-4a90-a9ab-8c8354d71d01 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:49.895057 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-aaddf6be-dc95-4a90-a9ab-8c8354d71d01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:50.001475 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaddf6be-dc95-4a90-a9ab-8c8354d71d01 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:50.006072 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4542012b-105d-42c8-aee9-8ae3484159e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24d5e2637b7043e08313909d5b06e329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:50.007094 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4542012b-105d-42c8-aee9-8ae3484159e6 service cinder] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:50.007590 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4542012b-105d-42c8-aee9-8ae3484159e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:50.008440 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4542012b-105d-42c8-aee9-8ae3484159e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:50.022327 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4542012b-105d-42c8-aee9-8ae3484159e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:50.023927 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4542012b-105d-42c8-aee9-8ae3484159e6 service cinder] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:50.031789 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9879819-3c50-4de7-9fdc-7208f25a65ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:50.033910 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9879819-3c50-4de7-9fdc-7208f25a65ae service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:50.037858 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 949/1922] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:44:49 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 17:44:50.064123 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 974/1923] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:44:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:44:50.103969 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4542012b-105d-42c8-aee9-8ae3484159e6 service cinder] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:50.105477 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-4542012b-105d-42c8-aee9-8ae3484159e6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:50.120866 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4542012b-105d-42c8-aee9-8ae3484159e6 service cinder] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:50.127527 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8fcd57-4ab3-4d08-861d-789c677ff478 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24d5e2637b7043e08313909d5b06e329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:50.128203 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8fcd57-4ab3-4d08-861d-789c677ff478 service cinder] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:50.128530 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8fcd57-4ab3-4d08-861d-789c677ff478 service cinder] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:50.128811 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8fcd57-4ab3-4d08-861d-789c677ff478 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:50.133419 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-ad8fcd57-4ab3-4d08-861d-789c677ff478 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:50.453790 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8fcd57-4ab3-4d08-861d-789c677ff478 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:50.455658 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8fcd57-4ab3-4d08-861d-789c677ff478 service cinder] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:50.460211 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9879819-3c50-4de7-9fdc-7208f25a65ae service neutron] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:50.461636 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-a9879819-3c50-4de7-9fdc-7208f25a65ae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:50.475532 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 950/1924] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:44:50 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:44:50.486762 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9879819-3c50-4de7-9fdc-7208f25a65ae service neutron] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:50.498196 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27671d92-c5ee-49e2-90c7-80e95b6d9c70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:50.499713 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27671d92-c5ee-49e2-90c7-80e95b6d9c70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:50.501235 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27671d92-c5ee-49e2-90c7-80e95b6d9c70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:50.501528 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27671d92-c5ee-49e2-90c7-80e95b6d9c70 admin admin] PATH_INFO: `/v3/projects/8f016f5f2dea421d8cbc52c838ab1edd` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:50.515468 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27671d92-c5ee-49e2-90c7-80e95b6d9c70 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f016f5f2dea421d8cbc52c838ab1edd)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:50.517520 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27671d92-c5ee-49e2-90c7-80e95b6d9c70 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:50.607592 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-27671d92-c5ee-49e2-90c7-80e95b6d9c70 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f016f5f2dea421d8cbc52c838ab1edd', 'request_id': 'req-27671d92-c5ee-49e2-90c7-80e95b6d9c70'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:50.618109 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-27671d92-c5ee-49e2-90c7-80e95b6d9c70 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f016f5f2dea421d8cbc52c838ab1edd', 'request_id': 'req-27671d92-c5ee-49e2-90c7-80e95b6d9c70'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:50.629713 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-27671d92-c5ee-49e2-90c7-80e95b6d9c70 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f016f5f2dea421d8cbc52c838ab1edd', 'request_id': 'req-27671d92-c5ee-49e2-90c7-80e95b6d9c70'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:50.649601 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-27671d92-c5ee-49e2-90c7-80e95b6d9c70 admin admin] The token cache is being invalidate because project 8f016f5f2dea421d8cbc52c838ab1edd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:44:50.650682 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-27671d92-c5ee-49e2-90c7-80e95b6d9c70 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:50.651425 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-27671d92-c5ee-49e2-90c7-80e95b6d9c70 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:50.658577 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 975/1925] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jun 16 17:44:50 2022] DELETE /identity/v3/projects/8f016f5f2dea421d8cbc52c838ab1edd => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:44:50.660223 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbfadf9-c77f-46be-8aa0-887de66fc1c1 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:50.660565 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbfadf9-c77f-46be-8aa0-887de66fc1c1 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:50.660843 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbfadf9-c77f-46be-8aa0-887de66fc1c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:50.708962 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-7bbfadf9-c77f-46be-8aa0-887de66fc1c1 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:44:50.710510 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-7bbfadf9-c77f-46be-8aa0-887de66fc1c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:50.829238 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 951/1926] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:44:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:44:50.836906 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e54b244-c4a7-441a-8bb2-b5ce1938cda4 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:50.837720 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e54b244-c4a7-441a-8bb2-b5ce1938cda4 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:50.838152 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e54b244-c4a7-441a-8bb2-b5ce1938cda4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:50.865985 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-9e54b244-c4a7-441a-8bb2-b5ce1938cda4 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:44:50.867732 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-9e54b244-c4a7-441a-8bb2-b5ce1938cda4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:50.892118 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 976/1927] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:44:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:44:50.896714 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbfadf9-c77f-46be-8aa0-887de66fc1c1 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:50.910751 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbfadf9-c77f-46be-8aa0-887de66fc1c1 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:50.915620 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e5b995-8e8a-4b58-a249-65acc93ec96e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:50.916364 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e5b995-8e8a-4b58-a249-65acc93ec96e admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:50.916752 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e5b995-8e8a-4b58-a249-65acc93ec96e admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:50.917100 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e5b995-8e8a-4b58-a249-65acc93ec96e admin admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:50.917901 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e5b995-8e8a-4b58-a249-65acc93ec96e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:44:50.918275 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e5b995-8e8a-4b58-a249-65acc93ec96e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:50.920063 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e5b995-8e8a-4b58-a249-65acc93ec96e admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:50.936451 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 952/1928] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jun 16 17:44:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:44:50.939000 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e54b244-c4a7-441a-8bb2-b5ce1938cda4 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:50.950460 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e54b244-c4a7-441a-8bb2-b5ce1938cda4 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:50.956271 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e1042d-4eb5-40d8-95ea-e996160c6314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:50.957657 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e1042d-4eb5-40d8-95ea-e996160c6314 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:50.958035 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e1042d-4eb5-40d8-95ea-e996160c6314 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:50.958517 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e1042d-4eb5-40d8-95ea-e996160c6314 admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:50.959310 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e1042d-4eb5-40d8-95ea-e996160c6314 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:50.961485 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e1042d-4eb5-40d8-95ea-e996160c6314 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:51.001345 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 977/1929] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jun 16 17:44:50 2022] POST /identity/v3/projects => generated 377 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:44:51.005485 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e5b995-8e8a-4b58-a249-65acc93ec96e admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:51.018682 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e5b995-8e8a-4b58-a249-65acc93ec96e admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:51.030148 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:44:51.118468 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369d0131-22d1-4d8b-8873-7f6af05c6b45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:51.119610 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369d0131-22d1-4d8b-8873-7f6af05c6b45 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:51.120275 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369d0131-22d1-4d8b-8873-7f6af05c6b45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:51.120824 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369d0131-22d1-4d8b-8873-7f6af05c6b45 admin admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:51.122095 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369d0131-22d1-4d8b-8873-7f6af05c6b45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:51.124838 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369d0131-22d1-4d8b-8873-7f6af05c6b45 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:51.151911 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 953/1930] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jun 16 17:44:51 2022] POST /identity/v3/users => generated 289 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:44:51.156911 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e1042d-4eb5-40d8-95ea-e996160c6314 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:51.167323 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e1042d-4eb5-40d8-95ea-e996160c6314 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:51.172270 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757ff981-d115-498d-9f32-7da726f88c23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:51.172977 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757ff981-d115-498d-9f32-7da726f88c23 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:51.173256 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757ff981-d115-498d-9f32-7da726f88c23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:51.173526 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757ff981-d115-498d-9f32-7da726f88c23 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:51.174240 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757ff981-d115-498d-9f32-7da726f88c23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:51.175826 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757ff981-d115-498d-9f32-7da726f88c23 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:51.186966 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 978/1931] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:44:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:44:51.191031 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369d0131-22d1-4d8b-8873-7f6af05c6b45 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:51.202369 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369d0131-22d1-4d8b-8873-7f6af05c6b45 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:51.207565 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b25ff2-725d-483c-ab13-6449c30b3f8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:51.208238 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b25ff2-725d-483c-ab13-6449c30b3f8c admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:51.208559 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b25ff2-725d-483c-ab13-6449c30b3f8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:51.208834 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b25ff2-725d-483c-ab13-6449c30b3f8c admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:51.209562 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b25ff2-725d-483c-ab13-6449c30b3f8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:51.211175 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b25ff2-725d-483c-ab13-6449c30b3f8c admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:51.222490 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 954/1932] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:44:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:44:51.226644 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757ff981-d115-498d-9f32-7da726f88c23 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:51.232020 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:44:51.238318 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757ff981-d115-498d-9f32-7da726f88c23 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:51.244514 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f98b90-8a0d-49ff-9a58-2ef1317b8d01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:51.245970 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f98b90-8a0d-49ff-9a58-2ef1317b8d01 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:51.246267 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f98b90-8a0d-49ff-9a58-2ef1317b8d01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:51.246641 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f98b90-8a0d-49ff-9a58-2ef1317b8d01 admin admin] PATH_INFO: `/v3/projects/b6bf545283e949579742168b582f1b15/users/6b670c20f0814cb3bc34b39731875e56/roles/0be2075946f54c8485f1c275f746478a` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:51.264212 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f98b90-8a0d-49ff-9a58-2ef1317b8d01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6bf545283e949579742168b582f1b15, user_id=6b670c20f0814cb3bc34b39731875e56, role_id=0be2075946f54c8485f1c275f746478a)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:51.267111 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f98b90-8a0d-49ff-9a58-2ef1317b8d01 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:51.283980 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 979/1933] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:44:51 2022] PUT /identity/v3/projects/b6bf545283e949579742168b582f1b15/users/6b670c20f0814cb3bc34b39731875e56/roles/0be2075946f54c8485f1c275f746478a => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:44:51.294469 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d2049f-520d-4244-b953-1618d29d33ce None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:51.294890 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d2049f-520d-4244-b953-1618d29d33ce None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:51.295089 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d2049f-520d-4244-b953-1618d29d33ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:51.323992 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-38d2049f-520d-4244-b953-1618d29d33ce None None] MFA Rules not processed for user `6b670c20f0814cb3bc34b39731875e56`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:44:51.325383 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-38d2049f-520d-4244-b953-1618d29d33ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:51.393199 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 955/1934] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:44:51 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:44:51.397855 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f98b90-8a0d-49ff-9a58-2ef1317b8d01 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:51.511720 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f98b90-8a0d-49ff-9a58-2ef1317b8d01 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:51.517553 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712abd5b-3a1a-4fef-9c32-da6839c694fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:51.518305 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712abd5b-3a1a-4fef-9c32-da6839c694fd admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:51.518659 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712abd5b-3a1a-4fef-9c32-da6839c694fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:51.519011 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712abd5b-3a1a-4fef-9c32-da6839c694fd admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:51.519799 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712abd5b-3a1a-4fef-9c32-da6839c694fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:51.521814 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712abd5b-3a1a-4fef-9c32-da6839c694fd admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:51.549844 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 980/1935] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jun 16 17:44:51 2022] POST /identity/v3/projects => generated 377 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:44:51.555683 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d2049f-520d-4244-b953-1618d29d33ce None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:51.567367 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d2049f-520d-4244-b953-1618d29d33ce None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:51.644490 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a634f3d-109b-44a3-80f0-5eb449fbdbfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:51.645100 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a634f3d-109b-44a3-80f0-5eb449fbdbfe admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:51.645380 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a634f3d-109b-44a3-80f0-5eb449fbdbfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:51.645652 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a634f3d-109b-44a3-80f0-5eb449fbdbfe admin admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:51.646360 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a634f3d-109b-44a3-80f0-5eb449fbdbfe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:51.648552 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a634f3d-109b-44a3-80f0-5eb449fbdbfe admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:51.671971 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 956/1936] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jun 16 17:44:51 2022] POST /identity/v3/users => generated 289 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:44:51.673267 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712abd5b-3a1a-4fef-9c32-da6839c694fd admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:51.684236 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712abd5b-3a1a-4fef-9c32-da6839c694fd admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:51.689830 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18a8d42-a990-4fa7-9c50-664634d450cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:51.690734 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18a8d42-a990-4fa7-9c50-664634d450cb admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:51.691404 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18a8d42-a990-4fa7-9c50-664634d450cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:51.691680 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18a8d42-a990-4fa7-9c50-664634d450cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:51.692510 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18a8d42-a990-4fa7-9c50-664634d450cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:51.694147 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18a8d42-a990-4fa7-9c50-664634d450cb admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:51.706124 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 981/1937] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:44:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:44:51.710402 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a634f3d-109b-44a3-80f0-5eb449fbdbfe admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:51.721825 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a634f3d-109b-44a3-80f0-5eb449fbdbfe admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:51.727218 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4384980a-09e8-4397-a348-419eb8678bc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:51.727975 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4384980a-09e8-4397-a348-419eb8678bc7 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:51.728239 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4384980a-09e8-4397-a348-419eb8678bc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:51.728546 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4384980a-09e8-4397-a348-419eb8678bc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:51.729269 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4384980a-09e8-4397-a348-419eb8678bc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:51.730856 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4384980a-09e8-4397-a348-419eb8678bc7 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:51.734096 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:44:51.743728 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 957/1938] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:44:51 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:44:51.748027 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18a8d42-a990-4fa7-9c50-664634d450cb admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:51.758918 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18a8d42-a990-4fa7-9c50-664634d450cb admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:51.763994 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b912caf1-21ec-432d-90ce-679c36501aca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:51.764918 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b912caf1-21ec-432d-90ce-679c36501aca admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:51.765197 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b912caf1-21ec-432d-90ce-679c36501aca admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:51.765468 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b912caf1-21ec-432d-90ce-679c36501aca admin admin] PATH_INFO: `/v3/projects/2295c03faa4a4e689d5711575e0e314b/users/cbb7eda90b65497c8f42e25cc458b86c/roles/0be2075946f54c8485f1c275f746478a` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:51.792359 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b912caf1-21ec-432d-90ce-679c36501aca admin admin] RBAC: Authorizing `identity:create_grant(project_id=2295c03faa4a4e689d5711575e0e314b, user_id=cbb7eda90b65497c8f42e25cc458b86c, role_id=0be2075946f54c8485f1c275f746478a)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:51.797875 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b912caf1-21ec-432d-90ce-679c36501aca admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:51.818709 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 982/1939] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:44:51 2022] PUT /identity/v3/projects/2295c03faa4a4e689d5711575e0e314b/users/cbb7eda90b65497c8f42e25cc458b86c/roles/0be2075946f54c8485f1c275f746478a => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:44:51.830099 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68cb1f9-51d4-453f-ab7a-f73a146e277f None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:51.830612 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68cb1f9-51d4-453f-ab7a-f73a146e277f None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:51.830823 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68cb1f9-51d4-453f-ab7a-f73a146e277f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:51.859447 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-e68cb1f9-51d4-453f-ab7a-f73a146e277f None None] MFA Rules not processed for user `cbb7eda90b65497c8f42e25cc458b86c`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:44:51.861146 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-e68cb1f9-51d4-453f-ab7a-f73a146e277f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:51.943992 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 958/1940] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:44:51 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:44:51.948670 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b912caf1-21ec-432d-90ce-679c36501aca admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:52.026710 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b912caf1-21ec-432d-90ce-679c36501aca admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:52.032055 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022271e7-7b92-41f2-a1ed-bb91f392808e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:52.032945 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022271e7-7b92-41f2-a1ed-bb91f392808e admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:52.033239 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022271e7-7b92-41f2-a1ed-bb91f392808e admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:52.033519 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022271e7-7b92-41f2-a1ed-bb91f392808e admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:52.034621 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022271e7-7b92-41f2-a1ed-bb91f392808e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:52.036322 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022271e7-7b92-41f2-a1ed-bb91f392808e admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:52.072681 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 983/1941] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jun 16 17:44:51 2022] POST /identity/v3/projects => generated 375 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:44:52.074017 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68cb1f9-51d4-453f-ab7a-f73a146e277f None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:52.085924 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68cb1f9-51d4-453f-ab7a-f73a146e277f None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:52.176421 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b27ffd5-1b99-4f3a-a89c-0db506d6b191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:52.176421 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b27ffd5-1b99-4f3a-a89c-0db506d6b191 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:52.176421 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b27ffd5-1b99-4f3a-a89c-0db506d6b191 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:52.176748 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b27ffd5-1b99-4f3a-a89c-0db506d6b191 admin admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:52.177227 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b27ffd5-1b99-4f3a-a89c-0db506d6b191 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:52.179325 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b27ffd5-1b99-4f3a-a89c-0db506d6b191 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:52.210281 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 959/1942] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jun 16 17:44:52 2022] POST /identity/v3/users => generated 294 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:44:52.211630 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022271e7-7b92-41f2-a1ed-bb91f392808e admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:52.224656 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022271e7-7b92-41f2-a1ed-bb91f392808e admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:52.233414 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c873ce0-5624-41ab-a442-0d45ac6e47d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:52.234200 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c873ce0-5624-41ab-a442-0d45ac6e47d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:52.234491 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c873ce0-5624-41ab-a442-0d45ac6e47d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:52.234771 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c873ce0-5624-41ab-a442-0d45ac6e47d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:52.235509 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c873ce0-5624-41ab-a442-0d45ac6e47d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:52.237446 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c873ce0-5624-41ab-a442-0d45ac6e47d2 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:52.250444 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 984/1943] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:44:52 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:44:52.254669 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b27ffd5-1b99-4f3a-a89c-0db506d6b191 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:52.267361 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b27ffd5-1b99-4f3a-a89c-0db506d6b191 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:52.272614 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fbec05-c70a-4252-a827-6dea2d701b2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:52.273579 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fbec05-c70a-4252-a827-6dea2d701b2f admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:52.273934 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fbec05-c70a-4252-a827-6dea2d701b2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:52.274280 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fbec05-c70a-4252-a827-6dea2d701b2f admin admin] PATH_INFO: `/v3/projects/8052e84593eb4fe8a1803fa7b4f99ea1/users/ea9ded040f5d4b9dbd67d0983b0a1cdd/roles/3ca2dcbbd29d4a31a3ab78f2299bc239` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:52.296521 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fbec05-c70a-4252-a827-6dea2d701b2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8052e84593eb4fe8a1803fa7b4f99ea1, user_id=ea9ded040f5d4b9dbd67d0983b0a1cdd, role_id=3ca2dcbbd29d4a31a3ab78f2299bc239)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:52.299482 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fbec05-c70a-4252-a827-6dea2d701b2f admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:52.316424 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 960/1944] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:44:52 2022] PUT /identity/v3/projects/8052e84593eb4fe8a1803fa7b4f99ea1/users/ea9ded040f5d4b9dbd67d0983b0a1cdd/roles/3ca2dcbbd29d4a31a3ab78f2299bc239 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:44:52.320319 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c873ce0-5624-41ab-a442-0d45ac6e47d2 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:52.326405 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:44:52.358761 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c873ce0-5624-41ab-a442-0d45ac6e47d2 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:52.446216 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7820b629-7ac5-446c-a9c3-705821563110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:52.446950 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7820b629-7ac5-446c-a9c3-705821563110 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:52.447236 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7820b629-7ac5-446c-a9c3-705821563110 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:52.447517 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7820b629-7ac5-446c-a9c3-705821563110 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:52.448309 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7820b629-7ac5-446c-a9c3-705821563110 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:52.450547 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7820b629-7ac5-446c-a9c3-705821563110 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:52.465695 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 985/1945] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:44:52 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:44:52.467767 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fbec05-c70a-4252-a827-6dea2d701b2f admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:52.478782 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fbec05-c70a-4252-a827-6dea2d701b2f admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:52.484944 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37cd963-e84e-4e03-9104-d0deeb81a786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:52.485834 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37cd963-e84e-4e03-9104-d0deeb81a786 admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:52.486113 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37cd963-e84e-4e03-9104-d0deeb81a786 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:52.486598 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37cd963-e84e-4e03-9104-d0deeb81a786 admin admin] PATH_INFO: `/v3/projects/8052e84593eb4fe8a1803fa7b4f99ea1/users/ea9ded040f5d4b9dbd67d0983b0a1cdd/roles/3ca2dcbbd29d4a31a3ab78f2299bc239` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:52.488598 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37cd963-e84e-4e03-9104-d0deeb81a786 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8052e84593eb4fe8a1803fa7b4f99ea1, user_id=ea9ded040f5d4b9dbd67d0983b0a1cdd, role_id=3ca2dcbbd29d4a31a3ab78f2299bc239)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:52.491535 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37cd963-e84e-4e03-9104-d0deeb81a786 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:52.504117 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 961/1946] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:44:52 2022] PUT /identity/v3/projects/8052e84593eb4fe8a1803fa7b4f99ea1/users/ea9ded040f5d4b9dbd67d0983b0a1cdd/roles/3ca2dcbbd29d4a31a3ab78f2299bc239 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:44:52.515866 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f0d0e9-32c9-4d17-8bdc-38efca726a39 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:52.517047 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f0d0e9-32c9-4d17-8bdc-38efca726a39 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:52.517361 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f0d0e9-32c9-4d17-8bdc-38efca726a39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:52.555428 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-65f0d0e9-32c9-4d17-8bdc-38efca726a39 None None] MFA Rules not processed for user `ea9ded040f5d4b9dbd67d0983b0a1cdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:44:52.555428 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-65f0d0e9-32c9-4d17-8bdc-38efca726a39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:52.715324 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 986/1947] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:44:52 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:44:52.729093 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37cd963-e84e-4e03-9104-d0deeb81a786 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:52.731112 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-c37cd963-e84e-4e03-9104-d0deeb81a786 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:52.785067 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:44:52.806552 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37cd963-e84e-4e03-9104-d0deeb81a786 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:52.811250 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b91a92b-b548-4453-8638-fb15eaa892ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24d5e2637b7043e08313909d5b06e329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:52.812550 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b91a92b-b548-4453-8638-fb15eaa892ab service cinder] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:52.813287 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b91a92b-b548-4453-8638-fb15eaa892ab service cinder] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:52.814564 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b91a92b-b548-4453-8638-fb15eaa892ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:52.893596 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b91a92b-b548-4453-8638-fb15eaa892ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:52.895274 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b91a92b-b548-4453-8638-fb15eaa892ab service cinder] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:52.907422 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 962/1948] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:44:52 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:44:54.529560 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f0d0e9-32c9-4d17-8bdc-38efca726a39 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:54.531645 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-65f0d0e9-32c9-4d17-8bdc-38efca726a39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:54.640090 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f0d0e9-32c9-4d17-8bdc-38efca726a39 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:54.659912 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555352f1-0f89-42a9-98fe-665a36676170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:54.660943 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555352f1-0f89-42a9-98fe-665a36676170 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:54.661228 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555352f1-0f89-42a9-98fe-665a36676170 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:54.661504 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555352f1-0f89-42a9-98fe-665a36676170 admin admin] PATH_INFO: `/v3/users/8d53e94e50f54fd19ad53e8ac048a040` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:54.662733 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555352f1-0f89-42a9-98fe-665a36676170 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d53e94e50f54fd19ad53e8ac048a040)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:54.664682 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555352f1-0f89-42a9-98fe-665a36676170 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:54.697462 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:44:54.723548 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b91a92b-b548-4453-8638-fb15eaa892ab service cinder] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:54.725091 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-3b91a92b-b548-4453-8638-fb15eaa892ab service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:54.729634 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-555352f1-0f89-42a9-98fe-665a36676170 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d53e94e50f54fd19ad53e8ac048a040'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:54.739650 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b91a92b-b548-4453-8638-fb15eaa892ab service cinder] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:54.743664 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bc059f-f685-4d3a-bdb2-894267143f8c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24d5e2637b7043e08313909d5b06e329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:54.744337 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bc059f-f685-4d3a-bdb2-894267143f8c service cinder] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:54.744819 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bc059f-f685-4d3a-bdb2-894267143f8c service cinder] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:54.745261 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bc059f-f685-4d3a-bdb2-894267143f8c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:54.755819 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-555352f1-0f89-42a9-98fe-665a36676170 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d53e94e50f54fd19ad53e8ac048a040'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:54.756615 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-555352f1-0f89-42a9-98fe-665a36676170 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d53e94e50f54fd19ad53e8ac048a040'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:54.766846 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-555352f1-0f89-42a9-98fe-665a36676170 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d53e94e50f54fd19ad53e8ac048a040'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:54.782522 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-555352f1-0f89-42a9-98fe-665a36676170 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d53e94e50f54fd19ad53e8ac048a040'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:54.791963 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 987/1949] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jun 16 17:44:54 2022] DELETE /identity/v3/users/8d53e94e50f54fd19ad53e8ac048a040 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:44:54.798047 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048bac9a-4e8a-41a5-bf64-acb07c98f356 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:54.798780 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048bac9a-4e8a-41a5-bf64-acb07c98f356 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:54.799243 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048bac9a-4e8a-41a5-bf64-acb07c98f356 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:54.821180 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bc059f-f685-4d3a-bdb2-894267143f8c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:54.822841 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bc059f-f685-4d3a-bdb2-894267143f8c service cinder] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:54.824481 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-048bac9a-4e8a-41a5-bf64-acb07c98f356 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:44:54.826947 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-048bac9a-4e8a-41a5-bf64-acb07c98f356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:54.848816 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 963/1950] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:44:54 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 17:44:54.930777 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 988/1951] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:44:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:44:54.942945 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bc059f-f685-4d3a-bdb2-894267143f8c service cinder] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:54.944804 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-27bc059f-f685-4d3a-bdb2-894267143f8c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:55.049910 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bc059f-f685-4d3a-bdb2-894267143f8c service cinder] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:55.102596 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:44:55.109172 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211232cb-8155-499e-a6ba-0bb7952f42c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:55.109957 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211232cb-8155-499e-a6ba-0bb7952f42c7 service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:55.110364 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211232cb-8155-499e-a6ba-0bb7952f42c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:55.110644 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211232cb-8155-499e-a6ba-0bb7952f42c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:55.122114 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211232cb-8155-499e-a6ba-0bb7952f42c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:55.123995 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211232cb-8155-499e-a6ba-0bb7952f42c7 service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:55.136030 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 964/1952] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:44:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:44:55.539154 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048bac9a-4e8a-41a5-bf64-acb07c98f356 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:55.540724 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-048bac9a-4e8a-41a5-bf64-acb07c98f356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:55.577606 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048bac9a-4e8a-41a5-bf64-acb07c98f356 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:55.584848 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025541cb-e303-49de-8637-64b3bf2959bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:55.586455 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025541cb-e303-49de-8637-64b3bf2959bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:55.588346 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025541cb-e303-49de-8637-64b3bf2959bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:55.588861 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025541cb-e303-49de-8637-64b3bf2959bd admin admin] PATH_INFO: `/v3/projects/31e96b2099aa491c977c03f25f8bfd83` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:55.590431 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025541cb-e303-49de-8637-64b3bf2959bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=31e96b2099aa491c977c03f25f8bfd83)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:55.592821 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025541cb-e303-49de-8637-64b3bf2959bd admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:55.609598 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211232cb-8155-499e-a6ba-0bb7952f42c7 service neutron] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:55.611399 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-211232cb-8155-499e-a6ba-0bb7952f42c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:55.687374 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211232cb-8155-499e-a6ba-0bb7952f42c7 service neutron] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:55.689800 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-025541cb-e303-49de-8637-64b3bf2959bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31e96b2099aa491c977c03f25f8bfd83', 'request_id': 'req-025541cb-e303-49de-8637-64b3bf2959bd'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:55.701434 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-025541cb-e303-49de-8637-64b3bf2959bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31e96b2099aa491c977c03f25f8bfd83', 'request_id': 'req-025541cb-e303-49de-8637-64b3bf2959bd'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:55.713448 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-025541cb-e303-49de-8637-64b3bf2959bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31e96b2099aa491c977c03f25f8bfd83', 'request_id': 'req-025541cb-e303-49de-8637-64b3bf2959bd'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:55.733679 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-025541cb-e303-49de-8637-64b3bf2959bd admin admin] The token cache is being invalidate because project 31e96b2099aa491c977c03f25f8bfd83 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:44:55.733891 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-025541cb-e303-49de-8637-64b3bf2959bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:55.734670 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-025541cb-e303-49de-8637-64b3bf2959bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:55.739295 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 989/1953] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jun 16 17:44:55 2022] DELETE /identity/v3/projects/31e96b2099aa491c977c03f25f8bfd83 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:44:55.749985 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025541cb-e303-49de-8637-64b3bf2959bd admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:55.755450 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-025541cb-e303-49de-8637-64b3bf2959bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:55.778302 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c2e99a-5d8f-46d9-9741-9289006a025a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24d5e2637b7043e08313909d5b06e329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:55.779083 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c2e99a-5d8f-46d9-9741-9289006a025a service cinder] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:55.779598 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c2e99a-5d8f-46d9-9741-9289006a025a service cinder] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:55.780093 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c2e99a-5d8f-46d9-9741-9289006a025a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:55.783509 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-b6c2e99a-5d8f-46d9-9741-9289006a025a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:55.801244 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:44:55.859862 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025541cb-e303-49de-8637-64b3bf2959bd admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:55.865167 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9d6519-3471-4ee6-9932-bed9bd7dc5bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:55.866202 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9d6519-3471-4ee6-9932-bed9bd7dc5bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:55.866485 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9d6519-3471-4ee6-9932-bed9bd7dc5bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:55.866763 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9d6519-3471-4ee6-9932-bed9bd7dc5bc admin admin] PATH_INFO: `/v3/users/423f9ecab4ff4713b607d0aa94aca899` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:55.868114 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9d6519-3471-4ee6-9932-bed9bd7dc5bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=423f9ecab4ff4713b607d0aa94aca899)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:55.870195 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9d6519-3471-4ee6-9932-bed9bd7dc5bc admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:55.887179 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c2e99a-5d8f-46d9-9741-9289006a025a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:55.889601 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c2e99a-5d8f-46d9-9741-9289006a025a service cinder] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:55.903758 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 965/1954] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:44:55 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:44:55.927779 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ce9d6519-3471-4ee6-9932-bed9bd7dc5bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '423f9ecab4ff4713b607d0aa94aca899'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:55.944570 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ce9d6519-3471-4ee6-9932-bed9bd7dc5bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '423f9ecab4ff4713b607d0aa94aca899'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:55.945778 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ce9d6519-3471-4ee6-9932-bed9bd7dc5bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '423f9ecab4ff4713b607d0aa94aca899'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:55.957831 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ce9d6519-3471-4ee6-9932-bed9bd7dc5bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '423f9ecab4ff4713b607d0aa94aca899'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:55.969896 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ce9d6519-3471-4ee6-9932-bed9bd7dc5bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '423f9ecab4ff4713b607d0aa94aca899'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:55.987642 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 990/1955] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jun 16 17:44:55 2022] DELETE /identity/v3/users/423f9ecab4ff4713b607d0aa94aca899 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:44:56.320856 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c2e99a-5d8f-46d9-9741-9289006a025a service cinder] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:56.333621 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c2e99a-5d8f-46d9-9741-9289006a025a service cinder] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:56.389362 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:44:56.400448 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacf28f7-5169-4d85-85e1-e832d28b8c8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:56.400448 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacf28f7-5169-4d85-85e1-e832d28b8c8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:56.400448 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacf28f7-5169-4d85-85e1-e832d28b8c8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:56.400448 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacf28f7-5169-4d85-85e1-e832d28b8c8a admin admin] PATH_INFO: `/v3/projects/9b4f4c4df0f94237b1c4f1cfc5b9b53a` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:56.401501 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacf28f7-5169-4d85-85e1-e832d28b8c8a admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b4f4c4df0f94237b1c4f1cfc5b9b53a)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:56.403463 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacf28f7-5169-4d85-85e1-e832d28b8c8a admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:56.463106 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-eacf28f7-5169-4d85-85e1-e832d28b8c8a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b4f4c4df0f94237b1c4f1cfc5b9b53a', 'request_id': 'req-eacf28f7-5169-4d85-85e1-e832d28b8c8a'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:56.473553 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-eacf28f7-5169-4d85-85e1-e832d28b8c8a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b4f4c4df0f94237b1c4f1cfc5b9b53a', 'request_id': 'req-eacf28f7-5169-4d85-85e1-e832d28b8c8a'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:56.482890 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-eacf28f7-5169-4d85-85e1-e832d28b8c8a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b4f4c4df0f94237b1c4f1cfc5b9b53a', 'request_id': 'req-eacf28f7-5169-4d85-85e1-e832d28b8c8a'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:56.496613 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-eacf28f7-5169-4d85-85e1-e832d28b8c8a admin admin] The token cache is being invalidate because project 9b4f4c4df0f94237b1c4f1cfc5b9b53a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80357) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:44:56.496907 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-eacf28f7-5169-4d85-85e1-e832d28b8c8a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:56.498422 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-eacf28f7-5169-4d85-85e1-e832d28b8c8a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:44:56.500024 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 966/1956] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jun 16 17:44:56 2022] DELETE /identity/v3/projects/9b4f4c4df0f94237b1c4f1cfc5b9b53a => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:44:56.506077 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6795d6-af85-4031-a76f-b9841988b2df None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:56.506366 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6795d6-af85-4031-a76f-b9841988b2df None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:56.506644 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6795d6-af85-4031-a76f-b9841988b2df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:56.526369 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-bc6795d6-af85-4031-a76f-b9841988b2df None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:44:56.527582 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-bc6795d6-af85-4031-a76f-b9841988b2df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:56.599055 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 991/1957] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:44:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:44:56.608646 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd1bd82-68f4-4528-812a-367a6395537e None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:56.609105 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd1bd82-68f4-4528-812a-367a6395537e None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:56.609452 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd1bd82-68f4-4528-812a-367a6395537e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:56.629327 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-bbd1bd82-68f4-4528-812a-367a6395537e None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:44:56.630603 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-bbd1bd82-68f4-4528-812a-367a6395537e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:56.639864 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 967/1958] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:44:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:44:56.643931 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6795d6-af85-4031-a76f-b9841988b2df None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:56.655042 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6795d6-af85-4031-a76f-b9841988b2df None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:56.661249 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da88989-2599-4d1c-9581-af645bd0bfdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:56.661933 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da88989-2599-4d1c-9581-af645bd0bfdd admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:56.662208 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da88989-2599-4d1c-9581-af645bd0bfdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:56.662483 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da88989-2599-4d1c-9581-af645bd0bfdd admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:56.663215 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da88989-2599-4d1c-9581-af645bd0bfdd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:44:56.663551 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da88989-2599-4d1c-9581-af645bd0bfdd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:56.665155 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da88989-2599-4d1c-9581-af645bd0bfdd admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:56.679015 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 992/1959] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jun 16 17:44:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:44:56.683432 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd1bd82-68f4-4528-812a-367a6395537e None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:56.693399 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd1bd82-68f4-4528-812a-367a6395537e None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:56.698326 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4956ed77-8a3e-44e7-ade8-2089214def92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:56.699003 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4956ed77-8a3e-44e7-ade8-2089214def92 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:56.699283 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4956ed77-8a3e-44e7-ade8-2089214def92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:56.699554 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4956ed77-8a3e-44e7-ade8-2089214def92 admin admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:56.700255 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4956ed77-8a3e-44e7-ade8-2089214def92 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:56.702335 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4956ed77-8a3e-44e7-ade8-2089214def92 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:56.733372 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 968/1960] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jun 16 17:44:56 2022] POST /identity/v3/projects => generated 377 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:44:56.734961 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da88989-2599-4d1c-9581-af645bd0bfdd admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:56.754413 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da88989-2599-4d1c-9581-af645bd0bfdd admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:56.805570 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:44:56.843150 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64858105-dae2-4773-98ea-c79760439fc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:56.843835 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64858105-dae2-4773-98ea-c79760439fc5 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:56.844117 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64858105-dae2-4773-98ea-c79760439fc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:56.844418 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64858105-dae2-4773-98ea-c79760439fc5 admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:56.845121 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64858105-dae2-4773-98ea-c79760439fc5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:56.847603 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64858105-dae2-4773-98ea-c79760439fc5 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:56.899980 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 993/1961] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jun 16 17:44:56 2022] POST /identity/v3/users => generated 289 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:44:56.901223 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4956ed77-8a3e-44e7-ade8-2089214def92 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:56.914311 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4956ed77-8a3e-44e7-ade8-2089214def92 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:56.920204 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3f80b5-b66b-467d-a294-f3f8c4d17a38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:56.921056 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3f80b5-b66b-467d-a294-f3f8c4d17a38 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:56.921341 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3f80b5-b66b-467d-a294-f3f8c4d17a38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:56.921615 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3f80b5-b66b-467d-a294-f3f8c4d17a38 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:56.922982 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3f80b5-b66b-467d-a294-f3f8c4d17a38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:56.924669 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3f80b5-b66b-467d-a294-f3f8c4d17a38 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:56.938716 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 969/1962] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:44:56 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:44:56.939884 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64858105-dae2-4773-98ea-c79760439fc5 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:56.950022 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64858105-dae2-4773-98ea-c79760439fc5 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:56.957592 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1623d1f-884f-4db6-98fe-a0ec74d1e3e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:56.958647 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1623d1f-884f-4db6-98fe-a0ec74d1e3e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:56.958849 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1623d1f-884f-4db6-98fe-a0ec74d1e3e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:56.959369 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1623d1f-884f-4db6-98fe-a0ec74d1e3e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:56.960303 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1623d1f-884f-4db6-98fe-a0ec74d1e3e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:56.963048 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1623d1f-884f-4db6-98fe-a0ec74d1e3e6 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:56.978056 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 994/1963] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:44:56 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:44:56.983027 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3f80b5-b66b-467d-a294-f3f8c4d17a38 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:56.993802 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3f80b5-b66b-467d-a294-f3f8c4d17a38 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:56.999260 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58188c10-2116-4add-a075-23dabcb58b3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:57.000321 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58188c10-2116-4add-a075-23dabcb58b3d admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:57.003959 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58188c10-2116-4add-a075-23dabcb58b3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:57.004504 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58188c10-2116-4add-a075-23dabcb58b3d admin admin] PATH_INFO: `/v3/projects/e54338527ec0437f8d0af022609e8537/users/307d03fa960541ae928464a7f55f8cdf/roles/0be2075946f54c8485f1c275f746478a` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:57.048605 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58188c10-2116-4add-a075-23dabcb58b3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e54338527ec0437f8d0af022609e8537, user_id=307d03fa960541ae928464a7f55f8cdf, role_id=0be2075946f54c8485f1c275f746478a)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:57.053339 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58188c10-2116-4add-a075-23dabcb58b3d admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:57.090030 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 970/1964] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:44:56 2022] PUT /identity/v3/projects/e54338527ec0437f8d0af022609e8537/users/307d03fa960541ae928464a7f55f8cdf/roles/0be2075946f54c8485f1c275f746478a => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:44:57.094126 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61203344-bbfe-49ce-adf4-9bb52a238084 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:57.094424 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61203344-bbfe-49ce-adf4-9bb52a238084 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:57.094699 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61203344-bbfe-49ce-adf4-9bb52a238084 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:57.156990 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-61203344-bbfe-49ce-adf4-9bb52a238084 None None] MFA Rules not processed for user `307d03fa960541ae928464a7f55f8cdf`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:44:57.158229 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-61203344-bbfe-49ce-adf4-9bb52a238084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:57.287048 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 995/1965] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:44:57 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:44:57.300093 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58188c10-2116-4add-a075-23dabcb58b3d admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:44:57.301916 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-58188c10-2116-4add-a075-23dabcb58b3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:44:57.392414 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58188c10-2116-4add-a075-23dabcb58b3d admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:44:57.399455 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b43f3b0-0685-4dc6-8e0e-a252a9b59d7d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24d5e2637b7043e08313909d5b06e329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:44:57.400699 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b43f3b0-0685-4dc6-8e0e-a252a9b59d7d service cinder] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:44:57.400985 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b43f3b0-0685-4dc6-8e0e-a252a9b59d7d service cinder] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:44:57.401694 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b43f3b0-0685-4dc6-8e0e-a252a9b59d7d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:44:57.415411 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b43f3b0-0685-4dc6-8e0e-a252a9b59d7d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:44:57.417829 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b43f3b0-0685-4dc6-8e0e-a252a9b59d7d service cinder] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:44:57.436509 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 971/1966] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:44:57 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:45:04.584955 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61203344-bbfe-49ce-adf4-9bb52a238084 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:04.587111 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-61203344-bbfe-49ce-adf4-9bb52a238084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:45:04.659432 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61203344-bbfe-49ce-adf4-9bb52a238084 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:04.664196 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f1fa59-e711-4558-a2da-c18dd5e930bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:04.665105 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f1fa59-e711-4558-a2da-c18dd5e930bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:04.665389 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f1fa59-e711-4558-a2da-c18dd5e930bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:04.665667 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f1fa59-e711-4558-a2da-c18dd5e930bd admin admin] PATH_INFO: `/v3/users/6b670c20f0814cb3bc34b39731875e56` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:04.666859 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f1fa59-e711-4558-a2da-c18dd5e930bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b670c20f0814cb3bc34b39731875e56)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:04.668912 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f1fa59-e711-4558-a2da-c18dd5e930bd admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:04.712099 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:45:04.714582 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-d2f1fa59-e711-4558-a2da-c18dd5e930bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b670c20f0814cb3bc34b39731875e56'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:04.728662 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-d2f1fa59-e711-4558-a2da-c18dd5e930bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b670c20f0814cb3bc34b39731875e56'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:04.729473 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-d2f1fa59-e711-4558-a2da-c18dd5e930bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b670c20f0814cb3bc34b39731875e56'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:04.737976 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-d2f1fa59-e711-4558-a2da-c18dd5e930bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b670c20f0814cb3bc34b39731875e56'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:04.746698 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-d2f1fa59-e711-4558-a2da-c18dd5e930bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b670c20f0814cb3bc34b39731875e56'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:04.753188 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 996/1967] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jun 16 17:45:04 2022] DELETE /identity/v3/users/6b670c20f0814cb3bc34b39731875e56 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:45:04.757808 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca794dff-733f-4d79-86e2-302b1fb700ed None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:04.758536 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca794dff-733f-4d79-86e2-302b1fb700ed None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:04.758627 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca794dff-733f-4d79-86e2-302b1fb700ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:04.775885 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-ca794dff-733f-4d79-86e2-302b1fb700ed None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:45:04.777116 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-ca794dff-733f-4d79-86e2-302b1fb700ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:45:04.798714 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:45:04.840164 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 972/1968] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:45:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:45:04.850248 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f1fa59-e711-4558-a2da-c18dd5e930bd admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:04.851688 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-d2f1fa59-e711-4558-a2da-c18dd5e930bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:45:04.912744 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f1fa59-e711-4558-a2da-c18dd5e930bd admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:04.916441 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90d435e-e3fc-454c-9d84-540831aa3bdf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:04.917088 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90d435e-e3fc-454c-9d84-540831aa3bdf service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:04.917371 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90d435e-e3fc-454c-9d84-540831aa3bdf service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:04.917647 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90d435e-e3fc-454c-9d84-540831aa3bdf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:04.927278 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90d435e-e3fc-454c-9d84-540831aa3bdf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:04.928747 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90d435e-e3fc-454c-9d84-540831aa3bdf service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:04.939290 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 997/1969] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:45:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:45:05.314055 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca794dff-733f-4d79-86e2-302b1fb700ed None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:05.315481 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-ca794dff-733f-4d79-86e2-302b1fb700ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:45:05.330551 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca794dff-733f-4d79-86e2-302b1fb700ed None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:05.335971 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2778c45-30b1-4025-9988-9fd4dcb80465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:05.337144 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2778c45-30b1-4025-9988-9fd4dcb80465 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:05.337504 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2778c45-30b1-4025-9988-9fd4dcb80465 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:05.337783 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2778c45-30b1-4025-9988-9fd4dcb80465 admin admin] PATH_INFO: `/v3/projects/b6bf545283e949579742168b582f1b15` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:05.339274 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2778c45-30b1-4025-9988-9fd4dcb80465 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6bf545283e949579742168b582f1b15)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:05.342078 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2778c45-30b1-4025-9988-9fd4dcb80465 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:05.395462 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-f2778c45-30b1-4025-9988-9fd4dcb80465 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6bf545283e949579742168b582f1b15', 'request_id': 'req-f2778c45-30b1-4025-9988-9fd4dcb80465'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:05.402710 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-f2778c45-30b1-4025-9988-9fd4dcb80465 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6bf545283e949579742168b582f1b15', 'request_id': 'req-f2778c45-30b1-4025-9988-9fd4dcb80465'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:05.408993 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-f2778c45-30b1-4025-9988-9fd4dcb80465 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6bf545283e949579742168b582f1b15', 'request_id': 'req-f2778c45-30b1-4025-9988-9fd4dcb80465'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:05.420069 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-f2778c45-30b1-4025-9988-9fd4dcb80465 admin admin] The token cache is being invalidate because project b6bf545283e949579742168b582f1b15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80357) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:45:05.420363 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-f2778c45-30b1-4025-9988-9fd4dcb80465 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:05.421131 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-f2778c45-30b1-4025-9988-9fd4dcb80465 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:05.422690 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 973/1970] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jun 16 17:45:05 2022] DELETE /identity/v3/projects/b6bf545283e949579742168b582f1b15 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:45:05.426754 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90d435e-e3fc-454c-9d84-540831aa3bdf service neutron] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:05.436505 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90d435e-e3fc-454c-9d84-540831aa3bdf service neutron] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:05.495608 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39c8c66-4d62-46be-a28d-16df73e5477b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:05.496625 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39c8c66-4d62-46be-a28d-16df73e5477b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:05.496907 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39c8c66-4d62-46be-a28d-16df73e5477b admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:05.497306 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39c8c66-4d62-46be-a28d-16df73e5477b admin admin] PATH_INFO: `/v3/users/cbb7eda90b65497c8f42e25cc458b86c` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:05.498589 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39c8c66-4d62-46be-a28d-16df73e5477b admin admin] RBAC: Authorizing `identity:delete_user(user_id=cbb7eda90b65497c8f42e25cc458b86c)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:05.500899 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39c8c66-4d62-46be-a28d-16df73e5477b admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:05.551093 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-c39c8c66-4d62-46be-a28d-16df73e5477b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cbb7eda90b65497c8f42e25cc458b86c'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:05.562925 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-c39c8c66-4d62-46be-a28d-16df73e5477b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cbb7eda90b65497c8f42e25cc458b86c'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:05.563630 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-c39c8c66-4d62-46be-a28d-16df73e5477b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cbb7eda90b65497c8f42e25cc458b86c'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:05.564874 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:45:05.572187 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-c39c8c66-4d62-46be-a28d-16df73e5477b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cbb7eda90b65497c8f42e25cc458b86c'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:05.580068 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-c39c8c66-4d62-46be-a28d-16df73e5477b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cbb7eda90b65497c8f42e25cc458b86c'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:05.586083 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 998/1971] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jun 16 17:45:05 2022] DELETE /identity/v3/users/cbb7eda90b65497c8f42e25cc458b86c => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:45:05.866144 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2778c45-30b1-4025-9988-9fd4dcb80465 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:05.868541 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-f2778c45-30b1-4025-9988-9fd4dcb80465 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:45:05.883818 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2778c45-30b1-4025-9988-9fd4dcb80465 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:05.942791 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9303c9-3ec9-4510-9337-227897c5649c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:05.943586 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9303c9-3ec9-4510-9337-227897c5649c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:05.943873 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9303c9-3ec9-4510-9337-227897c5649c admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:05.944152 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9303c9-3ec9-4510-9337-227897c5649c admin admin] PATH_INFO: `/v3/projects/2295c03faa4a4e689d5711575e0e314b` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:05.945397 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9303c9-3ec9-4510-9337-227897c5649c admin admin] RBAC: Authorizing `identity:delete_project(project_id=2295c03faa4a4e689d5711575e0e314b)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:05.947441 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9303c9-3ec9-4510-9337-227897c5649c admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:06.006447 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-ac9303c9-3ec9-4510-9337-227897c5649c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2295c03faa4a4e689d5711575e0e314b', 'request_id': 'req-ac9303c9-3ec9-4510-9337-227897c5649c'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:06.009285 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:45:06.016077 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-ac9303c9-3ec9-4510-9337-227897c5649c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2295c03faa4a4e689d5711575e0e314b', 'request_id': 'req-ac9303c9-3ec9-4510-9337-227897c5649c'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:06.023487 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-ac9303c9-3ec9-4510-9337-227897c5649c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2295c03faa4a4e689d5711575e0e314b', 'request_id': 'req-ac9303c9-3ec9-4510-9337-227897c5649c'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:06.035904 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-ac9303c9-3ec9-4510-9337-227897c5649c admin admin] The token cache is being invalidate because project 2295c03faa4a4e689d5711575e0e314b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80357) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:45:06.036191 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-ac9303c9-3ec9-4510-9337-227897c5649c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:06.036866 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-ac9303c9-3ec9-4510-9337-227897c5649c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:06.038423 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 974/1972] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jun 16 17:45:05 2022] DELETE /identity/v3/projects/2295c03faa4a4e689d5711575e0e314b => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:45:06.042614 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39c8c66-4d62-46be-a28d-16df73e5477b admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:06.044258 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-c39c8c66-4d62-46be-a28d-16df73e5477b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:45:06.057285 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39c8c66-4d62-46be-a28d-16df73e5477b admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:06.114846 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3317f98-bd89-47f7-a7ed-8fca26b3572e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:06.115722 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3317f98-bd89-47f7-a7ed-8fca26b3572e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:06.116004 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3317f98-bd89-47f7-a7ed-8fca26b3572e admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:06.116277 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3317f98-bd89-47f7-a7ed-8fca26b3572e admin admin] PATH_INFO: `/v3/users/ea9ded040f5d4b9dbd67d0983b0a1cdd` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:06.117492 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3317f98-bd89-47f7-a7ed-8fca26b3572e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea9ded040f5d4b9dbd67d0983b0a1cdd)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:06.119382 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3317f98-bd89-47f7-a7ed-8fca26b3572e admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:06.169922 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-f3317f98-bd89-47f7-a7ed-8fca26b3572e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea9ded040f5d4b9dbd67d0983b0a1cdd'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:06.181654 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-f3317f98-bd89-47f7-a7ed-8fca26b3572e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea9ded040f5d4b9dbd67d0983b0a1cdd'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:06.182333 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-f3317f98-bd89-47f7-a7ed-8fca26b3572e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea9ded040f5d4b9dbd67d0983b0a1cdd'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:06.190134 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-f3317f98-bd89-47f7-a7ed-8fca26b3572e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea9ded040f5d4b9dbd67d0983b0a1cdd'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:06.199639 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-f3317f98-bd89-47f7-a7ed-8fca26b3572e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea9ded040f5d4b9dbd67d0983b0a1cdd'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:06.206474 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 999/1973] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jun 16 17:45:06 2022] DELETE /identity/v3/users/ea9ded040f5d4b9dbd67d0983b0a1cdd => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:45:06.638900 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9303c9-3ec9-4510-9337-227897c5649c admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:06.640422 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-ac9303c9-3ec9-4510-9337-227897c5649c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:45:06.656054 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9303c9-3ec9-4510-9337-227897c5649c admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:06.722605 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a77170-d6fb-49d0-9299-90eec4e7da08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:06.723512 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a77170-d6fb-49d0-9299-90eec4e7da08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:06.723797 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a77170-d6fb-49d0-9299-90eec4e7da08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:06.724071 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a77170-d6fb-49d0-9299-90eec4e7da08 admin admin] PATH_INFO: `/v3/projects/8052e84593eb4fe8a1803fa7b4f99ea1` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:06.725292 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a77170-d6fb-49d0-9299-90eec4e7da08 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8052e84593eb4fe8a1803fa7b4f99ea1)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:06.727247 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a77170-d6fb-49d0-9299-90eec4e7da08 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:06.775073 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-b3a77170-d6fb-49d0-9299-90eec4e7da08 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8052e84593eb4fe8a1803fa7b4f99ea1', 'request_id': 'req-b3a77170-d6fb-49d0-9299-90eec4e7da08'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:06.783432 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-b3a77170-d6fb-49d0-9299-90eec4e7da08 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8052e84593eb4fe8a1803fa7b4f99ea1', 'request_id': 'req-b3a77170-d6fb-49d0-9299-90eec4e7da08'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:06.789004 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-b3a77170-d6fb-49d0-9299-90eec4e7da08 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8052e84593eb4fe8a1803fa7b4f99ea1', 'request_id': 'req-b3a77170-d6fb-49d0-9299-90eec4e7da08'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:06.800679 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-b3a77170-d6fb-49d0-9299-90eec4e7da08 admin admin] The token cache is being invalidate because project 8052e84593eb4fe8a1803fa7b4f99ea1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80357) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:45:06.800971 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-b3a77170-d6fb-49d0-9299-90eec4e7da08 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:06.801644 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-b3a77170-d6fb-49d0-9299-90eec4e7da08 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:06.803172 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 975/1974] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jun 16 17:45:06 2022] DELETE /identity/v3/projects/8052e84593eb4fe8a1803fa7b4f99ea1 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:45:06.808323 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d55b68-f03c-4ee9-a731-2aa1785d86b2 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:06.808820 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d55b68-f03c-4ee9-a731-2aa1785d86b2 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:06.808928 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d55b68-f03c-4ee9-a731-2aa1785d86b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:06.828466 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-e0d55b68-f03c-4ee9-a731-2aa1785d86b2 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:45:06.829669 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-e0d55b68-f03c-4ee9-a731-2aa1785d86b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:45:06.891893 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1000/1975] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:45:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:45:06.900795 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88da86b4-75a7-4d2a-b6c2-f9b4ec9b587d None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:06.901372 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88da86b4-75a7-4d2a-b6c2-f9b4ec9b587d None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:06.901479 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88da86b4-75a7-4d2a-b6c2-f9b4ec9b587d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:06.921424 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-88da86b4-75a7-4d2a-b6c2-f9b4ec9b587d None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:45:06.922631 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-88da86b4-75a7-4d2a-b6c2-f9b4ec9b587d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:45:06.930554 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 976/1976] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:45:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:45:06.934906 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d55b68-f03c-4ee9-a731-2aa1785d86b2 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:06.939758 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:45:06.946157 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d55b68-f03c-4ee9-a731-2aa1785d86b2 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:06.952565 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493244a4-9091-46ca-9c9d-d718224be753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:06.953242 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493244a4-9091-46ca-9c9d-d718224be753 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:06.953528 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493244a4-9091-46ca-9c9d-d718224be753 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:06.953809 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493244a4-9091-46ca-9c9d-d718224be753 admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:06.954533 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493244a4-9091-46ca-9c9d-d718224be753 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:45:06.954841 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493244a4-9091-46ca-9c9d-d718224be753 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:06.956464 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493244a4-9091-46ca-9c9d-d718224be753 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:06.968962 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1001/1977] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jun 16 17:45:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:45:06.973575 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88da86b4-75a7-4d2a-b6c2-f9b4ec9b587d None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:06.982998 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88da86b4-75a7-4d2a-b6c2-f9b4ec9b587d None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:06.987592 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0cdde1-df7e-47fa-8939-7342eecc257d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:06.988240 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0cdde1-df7e-47fa-8939-7342eecc257d admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:06.988556 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0cdde1-df7e-47fa-8939-7342eecc257d admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:06.988835 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0cdde1-df7e-47fa-8939-7342eecc257d admin admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:06.989515 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0cdde1-df7e-47fa-8939-7342eecc257d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:06.991398 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0cdde1-df7e-47fa-8939-7342eecc257d admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:07.017919 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 977/1978] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jun 16 17:45:06 2022] POST /identity/v3/projects => generated 367 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:45:07.022189 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493244a4-9091-46ca-9c9d-d718224be753 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:07.033685 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493244a4-9091-46ca-9c9d-d718224be753 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:07.089950 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07123f33-d4a6-4d3d-9fd0-c0e506bbd5ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:07.090614 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07123f33-d4a6-4d3d-9fd0-c0e506bbd5ff admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:07.091510 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07123f33-d4a6-4d3d-9fd0-c0e506bbd5ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:07.091803 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07123f33-d4a6-4d3d-9fd0-c0e506bbd5ff admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:07.092512 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07123f33-d4a6-4d3d-9fd0-c0e506bbd5ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:07.094398 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07123f33-d4a6-4d3d-9fd0-c0e506bbd5ff admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:07.113729 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1002/1979] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jun 16 17:45:07 2022] POST /identity/v3/users => generated 284 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:45:07.118266 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0cdde1-df7e-47fa-8939-7342eecc257d admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:07.140567 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0cdde1-df7e-47fa-8939-7342eecc257d admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:07.145053 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80c2386-3065-4bc0-9b9e-4d16f0cc1093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:07.145758 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80c2386-3065-4bc0-9b9e-4d16f0cc1093 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:07.146040 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80c2386-3065-4bc0-9b9e-4d16f0cc1093 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:07.146317 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80c2386-3065-4bc0-9b9e-4d16f0cc1093 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:07.147044 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80c2386-3065-4bc0-9b9e-4d16f0cc1093 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:07.148784 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80c2386-3065-4bc0-9b9e-4d16f0cc1093 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:07.161273 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 978/1980] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:45:07 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:45:07.165352 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07123f33-d4a6-4d3d-9fd0-c0e506bbd5ff admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:07.175686 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07123f33-d4a6-4d3d-9fd0-c0e506bbd5ff admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:07.180431 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7012c51-a499-43b2-9a90-0fcb14473be6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:07.181102 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7012c51-a499-43b2-9a90-0fcb14473be6 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:07.181417 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7012c51-a499-43b2-9a90-0fcb14473be6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:07.181696 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7012c51-a499-43b2-9a90-0fcb14473be6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:07.182451 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7012c51-a499-43b2-9a90-0fcb14473be6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:07.184100 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7012c51-a499-43b2-9a90-0fcb14473be6 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:07.194633 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1003/1981] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:45:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:45:07.198652 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80c2386-3065-4bc0-9b9e-4d16f0cc1093 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:07.209241 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80c2386-3065-4bc0-9b9e-4d16f0cc1093 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:07.213871 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e5805c-797c-4ac7-bf73-3156a7330823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:07.214746 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e5805c-797c-4ac7-bf73-3156a7330823 admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:07.215025 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e5805c-797c-4ac7-bf73-3156a7330823 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:07.215295 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e5805c-797c-4ac7-bf73-3156a7330823 admin admin] PATH_INFO: `/v3/projects/e82c0414b4b64fed8fdcaacbbd84fa31/users/1e29ecfb603b4a28aebab4aa5d8f6abe/roles/0be2075946f54c8485f1c275f746478a` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:07.230801 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e5805c-797c-4ac7-bf73-3156a7330823 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e82c0414b4b64fed8fdcaacbbd84fa31, user_id=1e29ecfb603b4a28aebab4aa5d8f6abe, role_id=0be2075946f54c8485f1c275f746478a)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:07.234076 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e5805c-797c-4ac7-bf73-3156a7330823 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:07.249667 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 979/1982] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:45:07 2022] PUT /identity/v3/projects/e82c0414b4b64fed8fdcaacbbd84fa31/users/1e29ecfb603b4a28aebab4aa5d8f6abe/roles/0be2075946f54c8485f1c275f746478a => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:45:07.259982 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302a9608-2e05-4b76-afc7-6bb759322b00 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:07.260277 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302a9608-2e05-4b76-afc7-6bb759322b00 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:07.260587 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302a9608-2e05-4b76-afc7-6bb759322b00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:07.287926 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-302a9608-2e05-4b76-afc7-6bb759322b00 None None] MFA Rules not processed for user `1e29ecfb603b4a28aebab4aa5d8f6abe`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:45:07.289162 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-302a9608-2e05-4b76-afc7-6bb759322b00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:45:07.353449 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1004/1983] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:45:07 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:45:07.365197 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e5805c-797c-4ac7-bf73-3156a7330823 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:07.366853 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-b9e5805c-797c-4ac7-bf73-3156a7330823 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:45:07.377220 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:DELETE=10,keystone:SELECT=87,keystone:INSERT=3 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:45:07.431836 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e5805c-797c-4ac7-bf73-3156a7330823 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:07.436558 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2c5fb3-a31c-4768-aa39-e87b5855ed13 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd89029c1b3134df1962c041988bead74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:07.437215 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2c5fb3-a31c-4768-aa39-e87b5855ed13 service glance] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:07.437496 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2c5fb3-a31c-4768-aa39-e87b5855ed13 service glance] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:07.437770 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2c5fb3-a31c-4768-aa39-e87b5855ed13 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:07.446424 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2c5fb3-a31c-4768-aa39-e87b5855ed13 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:07.447900 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2c5fb3-a31c-4768-aa39-e87b5855ed13 service glance] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:07.458154 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 980/1984] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:45:07 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:45:17.350519 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=71,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:45:17.453981 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=23 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:45:21.714373 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302a9608-2e05-4b76-afc7-6bb759322b00 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:21.716614 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-302a9608-2e05-4b76-afc7-6bb759322b00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:45:21.789529 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302a9608-2e05-4b76-afc7-6bb759322b00 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:21.794814 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ef8b47-5068-4e4b-8678-db150e968abc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:21.795686 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ef8b47-5068-4e4b-8678-db150e968abc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:21.795969 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ef8b47-5068-4e4b-8678-db150e968abc admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:21.796240 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ef8b47-5068-4e4b-8678-db150e968abc admin admin] PATH_INFO: `/v3/users/307d03fa960541ae928464a7f55f8cdf` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:21.797460 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ef8b47-5068-4e4b-8678-db150e968abc admin admin] RBAC: Authorizing `identity:delete_user(user_id=307d03fa960541ae928464a7f55f8cdf)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:21.799390 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ef8b47-5068-4e4b-8678-db150e968abc admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:21.848950 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-16ef8b47-5068-4e4b-8678-db150e968abc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '307d03fa960541ae928464a7f55f8cdf'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:21.871416 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-16ef8b47-5068-4e4b-8678-db150e968abc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '307d03fa960541ae928464a7f55f8cdf'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:21.872308 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-16ef8b47-5068-4e4b-8678-db150e968abc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '307d03fa960541ae928464a7f55f8cdf'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:21.880537 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-16ef8b47-5068-4e4b-8678-db150e968abc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '307d03fa960541ae928464a7f55f8cdf'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:21.891419 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-16ef8b47-5068-4e4b-8678-db150e968abc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '307d03fa960541ae928464a7f55f8cdf'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:21.898595 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1005/1985] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jun 16 17:45:21 2022] DELETE /identity/v3/users/307d03fa960541ae928464a7f55f8cdf => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:45:21.903460 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e172769b-9e77-4c5c-bb0e-85a255cc4b5e None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:21.904080 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e172769b-9e77-4c5c-bb0e-85a255cc4b5e None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:21.904080 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e172769b-9e77-4c5c-bb0e-85a255cc4b5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:21.930189 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-e172769b-9e77-4c5c-bb0e-85a255cc4b5e None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:45:21.932126 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-e172769b-9e77-4c5c-bb0e-85a255cc4b5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:45:21.999533 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 981/1986] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:45:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:45:22.010819 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ef8b47-5068-4e4b-8678-db150e968abc admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:22.012583 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-16ef8b47-5068-4e4b-8678-db150e968abc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:45:22.073797 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ef8b47-5068-4e4b-8678-db150e968abc admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:22.078244 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e54a85-f63f-48d9-97b1-90bb0fdb4189 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:22.078896 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e54a85-f63f-48d9-97b1-90bb0fdb4189 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:22.079181 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e54a85-f63f-48d9-97b1-90bb0fdb4189 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:22.079459 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e54a85-f63f-48d9-97b1-90bb0fdb4189 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:22.088516 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e54a85-f63f-48d9-97b1-90bb0fdb4189 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:22.089946 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e54a85-f63f-48d9-97b1-90bb0fdb4189 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:22.100223 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1006/1987] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:45:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:45:22.475874 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e172769b-9e77-4c5c-bb0e-85a255cc4b5e None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:22.477717 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-e172769b-9e77-4c5c-bb0e-85a255cc4b5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:45:22.492259 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e172769b-9e77-4c5c-bb0e-85a255cc4b5e None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:22.497295 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf5b3d8-e80c-4e2b-a298-765303db92a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:22.498560 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf5b3d8-e80c-4e2b-a298-765303db92a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:22.498847 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf5b3d8-e80c-4e2b-a298-765303db92a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:22.499212 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf5b3d8-e80c-4e2b-a298-765303db92a3 admin admin] PATH_INFO: `/v3/projects/e54338527ec0437f8d0af022609e8537` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:22.500452 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf5b3d8-e80c-4e2b-a298-765303db92a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e54338527ec0437f8d0af022609e8537)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:22.502454 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf5b3d8-e80c-4e2b-a298-765303db92a3 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:22.551022 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-faf5b3d8-e80c-4e2b-a298-765303db92a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e54338527ec0437f8d0af022609e8537', 'request_id': 'req-faf5b3d8-e80c-4e2b-a298-765303db92a3'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:22.558623 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-faf5b3d8-e80c-4e2b-a298-765303db92a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e54338527ec0437f8d0af022609e8537', 'request_id': 'req-faf5b3d8-e80c-4e2b-a298-765303db92a3'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:22.564214 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-faf5b3d8-e80c-4e2b-a298-765303db92a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e54338527ec0437f8d0af022609e8537', 'request_id': 'req-faf5b3d8-e80c-4e2b-a298-765303db92a3'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:22.575831 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-faf5b3d8-e80c-4e2b-a298-765303db92a3 admin admin] The token cache is being invalidate because project e54338527ec0437f8d0af022609e8537 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80357) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:45:22.576123 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-faf5b3d8-e80c-4e2b-a298-765303db92a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:22.576759 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-faf5b3d8-e80c-4e2b-a298-765303db92a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:22.578365 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 982/1988] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jun 16 17:45:22 2022] DELETE /identity/v3/projects/e54338527ec0437f8d0af022609e8537 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:45:22.583312 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25af6d5-96f7-41cb-9cb7-bed9a745fa89 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:22.583604 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25af6d5-96f7-41cb-9cb7-bed9a745fa89 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:22.583878 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25af6d5-96f7-41cb-9cb7-bed9a745fa89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:22.603095 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-b25af6d5-96f7-41cb-9cb7-bed9a745fa89 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:45:22.604295 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-b25af6d5-96f7-41cb-9cb7-bed9a745fa89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:45:22.639389 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:45:22.684850 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1007/1989] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:45:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:45:22.692350 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e96c88-6df3-42e1-85b5-94bc9ff2ca68 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:22.692669 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e96c88-6df3-42e1-85b5-94bc9ff2ca68 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:22.692948 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e96c88-6df3-42e1-85b5-94bc9ff2ca68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:22.711396 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-92e96c88-6df3-42e1-85b5-94bc9ff2ca68 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:45:22.712616 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-92e96c88-6df3-42e1-85b5-94bc9ff2ca68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:45:22.721857 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 983/1990] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:45:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:45:22.725911 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25af6d5-96f7-41cb-9cb7-bed9a745fa89 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:22.736419 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25af6d5-96f7-41cb-9cb7-bed9a745fa89 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:22.740897 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83626b8-0679-4ca9-9e02-f7135e2fc4a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:22.741547 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83626b8-0679-4ca9-9e02-f7135e2fc4a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:22.741827 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83626b8-0679-4ca9-9e02-f7135e2fc4a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:22.742098 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83626b8-0679-4ca9-9e02-f7135e2fc4a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:22.742810 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83626b8-0679-4ca9-9e02-f7135e2fc4a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:45:22.743111 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83626b8-0679-4ca9-9e02-f7135e2fc4a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:22.744716 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83626b8-0679-4ca9-9e02-f7135e2fc4a0 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:22.756399 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1008/1991] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jun 16 17:45:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:45:22.760875 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e96c88-6df3-42e1-85b5-94bc9ff2ca68 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:22.771168 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e96c88-6df3-42e1-85b5-94bc9ff2ca68 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:22.775767 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cd300a-e975-4d8a-9653-3b98bfc6265c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:22.776433 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cd300a-e975-4d8a-9653-3b98bfc6265c admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:22.776723 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cd300a-e975-4d8a-9653-3b98bfc6265c admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:22.776998 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cd300a-e975-4d8a-9653-3b98bfc6265c admin admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:22.778413 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cd300a-e975-4d8a-9653-3b98bfc6265c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:22.780817 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cd300a-e975-4d8a-9653-3b98bfc6265c admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:22.805478 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 984/1992] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jun 16 17:45:22 2022] POST /identity/v3/projects => generated 383 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:45:22.809716 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83626b8-0679-4ca9-9e02-f7135e2fc4a0 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:22.819549 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83626b8-0679-4ca9-9e02-f7135e2fc4a0 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:22.877914 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971c9316-c888-42aa-ada3-ad1c2aefe0ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:22.878589 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971c9316-c888-42aa-ada3-ad1c2aefe0ed admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:22.878873 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971c9316-c888-42aa-ada3-ad1c2aefe0ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:22.879148 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971c9316-c888-42aa-ada3-ad1c2aefe0ed admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:22.879836 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971c9316-c888-42aa-ada3-ad1c2aefe0ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:22.881775 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971c9316-c888-42aa-ada3-ad1c2aefe0ed admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:22.902573 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1009/1993] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jun 16 17:45:22 2022] POST /identity/v3/users => generated 292 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:45:22.907392 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cd300a-e975-4d8a-9653-3b98bfc6265c admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:22.917521 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cd300a-e975-4d8a-9653-3b98bfc6265c admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:22.922237 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d10f2c-8a02-4ea4-841d-5cfd985c47d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:22.922887 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d10f2c-8a02-4ea4-841d-5cfd985c47d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:22.923168 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d10f2c-8a02-4ea4-841d-5cfd985c47d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:22.923440 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d10f2c-8a02-4ea4-841d-5cfd985c47d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:22.924168 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d10f2c-8a02-4ea4-841d-5cfd985c47d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:22.925785 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d10f2c-8a02-4ea4-841d-5cfd985c47d6 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:22.939009 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 985/1994] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:45:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:45:22.942895 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971c9316-c888-42aa-ada3-ad1c2aefe0ed admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:22.958075 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971c9316-c888-42aa-ada3-ad1c2aefe0ed admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:22.962947 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74b8ba6-2758-4ec7-a172-c675225ca743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:22.963618 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74b8ba6-2758-4ec7-a172-c675225ca743 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:22.963904 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74b8ba6-2758-4ec7-a172-c675225ca743 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:22.964181 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74b8ba6-2758-4ec7-a172-c675225ca743 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:22.964964 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74b8ba6-2758-4ec7-a172-c675225ca743 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:22.966668 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74b8ba6-2758-4ec7-a172-c675225ca743 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:22.977428 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1010/1995] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:45:22 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:45:22.981798 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d10f2c-8a02-4ea4-841d-5cfd985c47d6 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:22.992210 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d10f2c-8a02-4ea4-841d-5cfd985c47d6 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:22.998104 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e537fb0e-1f7d-4130-8d4b-e2dd768c8a36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:22.999037 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e537fb0e-1f7d-4130-8d4b-e2dd768c8a36 admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:22.999323 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e537fb0e-1f7d-4130-8d4b-e2dd768c8a36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:22.999600 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e537fb0e-1f7d-4130-8d4b-e2dd768c8a36 admin admin] PATH_INFO: `/v3/projects/bed678c9985d4f249517d30769e64f1d/users/e78c9b1a1c7943509f234a7d1d6a6767/roles/0be2075946f54c8485f1c275f746478a` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:23.015748 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e537fb0e-1f7d-4130-8d4b-e2dd768c8a36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bed678c9985d4f249517d30769e64f1d, user_id=e78c9b1a1c7943509f234a7d1d6a6767, role_id=0be2075946f54c8485f1c275f746478a)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:23.018582 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e537fb0e-1f7d-4130-8d4b-e2dd768c8a36 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:23.032734 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 986/1996] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:45:22 2022] PUT /identity/v3/projects/bed678c9985d4f249517d30769e64f1d/users/e78c9b1a1c7943509f234a7d1d6a6767/roles/0be2075946f54c8485f1c275f746478a => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:45:23.043771 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637713f8-1ab3-435f-af7c-34be20699732 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:23.044272 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637713f8-1ab3-435f-af7c-34be20699732 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:23.044492 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637713f8-1ab3-435f-af7c-34be20699732 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:23.083469 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-637713f8-1ab3-435f-af7c-34be20699732 None None] MFA Rules not processed for user `e78c9b1a1c7943509f234a7d1d6a6767`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:45:23.084886 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-637713f8-1ab3-435f-af7c-34be20699732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:45:23.162051 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1011/1997] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:45:23 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:45:23.166767 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e537fb0e-1f7d-4130-8d4b-e2dd768c8a36 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:23.189450 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:45:23.248338 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e537fb0e-1f7d-4130-8d4b-e2dd768c8a36 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:23.253185 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05eea505-7814-4101-8101-e99a5e987caf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:23.253858 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eea505-7814-4101-8101-e99a5e987caf admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:23.254145 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eea505-7814-4101-8101-e99a5e987caf admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:23.254454 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eea505-7814-4101-8101-e99a5e987caf admin admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:23.255160 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05eea505-7814-4101-8101-e99a5e987caf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:23.257152 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05eea505-7814-4101-8101-e99a5e987caf admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:23.287283 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 987/1998] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jun 16 17:45:23 2022] POST /identity/v3/projects => generated 383 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:45:23.288739 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637713f8-1ab3-435f-af7c-34be20699732 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:23.301416 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637713f8-1ab3-435f-af7c-34be20699732 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:23.315141 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:45:23.360830 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bec7cb-c9b9-4d96-80bf-a035f5a0348e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:23.361512 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bec7cb-c9b9-4d96-80bf-a035f5a0348e admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:23.361795 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bec7cb-c9b9-4d96-80bf-a035f5a0348e admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:23.362073 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bec7cb-c9b9-4d96-80bf-a035f5a0348e admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:23.363271 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bec7cb-c9b9-4d96-80bf-a035f5a0348e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:23.365257 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bec7cb-c9b9-4d96-80bf-a035f5a0348e admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:23.391824 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1012/1999] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jun 16 17:45:23 2022] POST /identity/v3/users => generated 292 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:45:23.393147 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05eea505-7814-4101-8101-e99a5e987caf admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:23.404950 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05eea505-7814-4101-8101-e99a5e987caf admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:23.409821 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b582d7d6-0ebb-41a7-819e-e8318de8a1e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:23.410490 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b582d7d6-0ebb-41a7-819e-e8318de8a1e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:23.410780 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b582d7d6-0ebb-41a7-819e-e8318de8a1e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:23.411063 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b582d7d6-0ebb-41a7-819e-e8318de8a1e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:23.411826 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b582d7d6-0ebb-41a7-819e-e8318de8a1e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:23.413548 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b582d7d6-0ebb-41a7-819e-e8318de8a1e2 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:23.430039 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 988/2000] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:45:23 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:45:23.431782 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bec7cb-c9b9-4d96-80bf-a035f5a0348e admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:23.447187 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bec7cb-c9b9-4d96-80bf-a035f5a0348e admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:23.454305 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa74eb06-1feb-407b-93ab-b210ffdfc57e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:23.456000 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa74eb06-1feb-407b-93ab-b210ffdfc57e admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:23.456210 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa74eb06-1feb-407b-93ab-b210ffdfc57e admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:23.456703 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa74eb06-1feb-407b-93ab-b210ffdfc57e admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:23.457527 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa74eb06-1feb-407b-93ab-b210ffdfc57e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:23.459308 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa74eb06-1feb-407b-93ab-b210ffdfc57e admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:23.471033 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1013/2001] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:45:23 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:45:23.476035 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b582d7d6-0ebb-41a7-819e-e8318de8a1e2 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:23.488429 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b582d7d6-0ebb-41a7-819e-e8318de8a1e2 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:23.493612 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee75e24-7586-4993-beac-7e3cac6530e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:23.494740 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee75e24-7586-4993-beac-7e3cac6530e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:23.495144 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee75e24-7586-4993-beac-7e3cac6530e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:23.495523 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee75e24-7586-4993-beac-7e3cac6530e4 admin admin] PATH_INFO: `/v3/projects/e4bb49572d1f41b6962bf7d1b052bf91/users/4b456bff3dd74f8d868ba9c2697ab086/roles/0be2075946f54c8485f1c275f746478a` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:23.515868 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee75e24-7586-4993-beac-7e3cac6530e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4bb49572d1f41b6962bf7d1b052bf91, user_id=4b456bff3dd74f8d868ba9c2697ab086, role_id=0be2075946f54c8485f1c275f746478a)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:23.518672 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee75e24-7586-4993-beac-7e3cac6530e4 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:23.537828 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 989/2002] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:45:23 2022] PUT /identity/v3/projects/e4bb49572d1f41b6962bf7d1b052bf91/users/4b456bff3dd74f8d868ba9c2697ab086/roles/0be2075946f54c8485f1c275f746478a => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:45:23.549718 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab62975-a019-461c-9257-f54345c11458 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:23.550088 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab62975-a019-461c-9257-f54345c11458 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:23.550447 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab62975-a019-461c-9257-f54345c11458 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:23.582439 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-3ab62975-a019-461c-9257-f54345c11458 None None] MFA Rules not processed for user `4b456bff3dd74f8d868ba9c2697ab086`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:45:23.583695 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-3ab62975-a019-461c-9257-f54345c11458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:45:23.651783 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1014/2003] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:45:23 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:45:23.657602 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee75e24-7586-4993-beac-7e3cac6530e4 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:23.729913 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee75e24-7586-4993-beac-7e3cac6530e4 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:23.734702 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383e3b49-c7bc-4c75-953f-6c9344353a8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:23.735371 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383e3b49-c7bc-4c75-953f-6c9344353a8d admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:23.735713 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383e3b49-c7bc-4c75-953f-6c9344353a8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:23.736013 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383e3b49-c7bc-4c75-953f-6c9344353a8d admin admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:23.736735 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383e3b49-c7bc-4c75-953f-6c9344353a8d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:23.738653 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383e3b49-c7bc-4c75-953f-6c9344353a8d admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:23.766440 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 990/2004] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jun 16 17:45:23 2022] POST /identity/v3/projects => generated 383 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:45:23.770180 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab62975-a019-461c-9257-f54345c11458 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:23.793666 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab62975-a019-461c-9257-f54345c11458 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:23.863695 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655f595e-7b08-4a35-a782-6cb5313c2bc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:23.864654 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655f595e-7b08-4a35-a782-6cb5313c2bc7 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:23.865018 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655f595e-7b08-4a35-a782-6cb5313c2bc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:23.865387 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655f595e-7b08-4a35-a782-6cb5313c2bc7 admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:23.866206 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655f595e-7b08-4a35-a782-6cb5313c2bc7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:23.868132 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655f595e-7b08-4a35-a782-6cb5313c2bc7 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:23.899743 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1015/2005] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jun 16 17:45:23 2022] POST /identity/v3/users => generated 298 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:45:23.901160 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383e3b49-c7bc-4c75-953f-6c9344353a8d admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:23.912136 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383e3b49-c7bc-4c75-953f-6c9344353a8d admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:23.917046 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e29b54-4f92-4f6b-b21a-bacb901dc550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:23.917717 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e29b54-4f92-4f6b-b21a-bacb901dc550 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:23.918006 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e29b54-4f92-4f6b-b21a-bacb901dc550 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:23.918319 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e29b54-4f92-4f6b-b21a-bacb901dc550 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:23.919445 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e29b54-4f92-4f6b-b21a-bacb901dc550 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:23.921117 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e29b54-4f92-4f6b-b21a-bacb901dc550 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:23.934917 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 991/2006] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:45:23 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:45:23.939060 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655f595e-7b08-4a35-a782-6cb5313c2bc7 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:23.950220 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655f595e-7b08-4a35-a782-6cb5313c2bc7 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:23.956638 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e36ef97-88af-4cf1-8308-a3fceb6c07f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:23.957530 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e36ef97-88af-4cf1-8308-a3fceb6c07f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:23.957814 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e36ef97-88af-4cf1-8308-a3fceb6c07f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:23.958091 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e36ef97-88af-4cf1-8308-a3fceb6c07f0 admin admin] PATH_INFO: `/v3/projects/b29bb73037b8496baaecbc9da5b21bca/users/4731e152aa994dffaa524b951c915bab/roles/3ca2dcbbd29d4a31a3ab78f2299bc239` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:23.970518 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:45:23.973875 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e36ef97-88af-4cf1-8308-a3fceb6c07f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b29bb73037b8496baaecbc9da5b21bca, user_id=4731e152aa994dffaa524b951c915bab, role_id=3ca2dcbbd29d4a31a3ab78f2299bc239)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:23.976956 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e36ef97-88af-4cf1-8308-a3fceb6c07f0 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:23.991632 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1016/2007] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:45:23 2022] PUT /identity/v3/projects/b29bb73037b8496baaecbc9da5b21bca/users/4731e152aa994dffaa524b951c915bab/roles/3ca2dcbbd29d4a31a3ab78f2299bc239 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:45:23.995409 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e29b54-4f92-4f6b-b21a-bacb901dc550 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:24.006181 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e29b54-4f92-4f6b-b21a-bacb901dc550 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:24.052962 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:45:24.070375 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670dfaea-0edb-4fd0-b462-8b3569bfdebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:24.071055 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670dfaea-0edb-4fd0-b462-8b3569bfdebb admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:24.071517 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670dfaea-0edb-4fd0-b462-8b3569bfdebb admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:24.071794 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670dfaea-0edb-4fd0-b462-8b3569bfdebb admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:24.072561 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670dfaea-0edb-4fd0-b462-8b3569bfdebb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:24.074329 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670dfaea-0edb-4fd0-b462-8b3569bfdebb admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:24.085365 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 992/2008] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:45:23 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 17:45:24.089748 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e36ef97-88af-4cf1-8308-a3fceb6c07f0 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:24.100757 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e36ef97-88af-4cf1-8308-a3fceb6c07f0 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:24.106277 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1f3f8a-768d-41b8-a65c-9e6a58d38ad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:24.107242 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1f3f8a-768d-41b8-a65c-9e6a58d38ad4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:24.107595 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1f3f8a-768d-41b8-a65c-9e6a58d38ad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:24.107989 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1f3f8a-768d-41b8-a65c-9e6a58d38ad4 admin admin] PATH_INFO: `/v3/projects/b29bb73037b8496baaecbc9da5b21bca/users/4731e152aa994dffaa524b951c915bab/roles/3ca2dcbbd29d4a31a3ab78f2299bc239` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:24.110253 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1f3f8a-768d-41b8-a65c-9e6a58d38ad4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b29bb73037b8496baaecbc9da5b21bca, user_id=4731e152aa994dffaa524b951c915bab, role_id=3ca2dcbbd29d4a31a3ab78f2299bc239)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:24.113077 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1f3f8a-768d-41b8-a65c-9e6a58d38ad4 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:24.144495 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1017/2009] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:45:24 2022] PUT /identity/v3/projects/b29bb73037b8496baaecbc9da5b21bca/users/4731e152aa994dffaa524b951c915bab/roles/3ca2dcbbd29d4a31a3ab78f2299bc239 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:45:24.157197 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23217d13-bc34-4414-8ece-c49e5947cc6a None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:24.157629 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23217d13-bc34-4414-8ece-c49e5947cc6a None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:24.158196 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23217d13-bc34-4414-8ece-c49e5947cc6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:24.195570 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-23217d13-bc34-4414-8ece-c49e5947cc6a None None] MFA Rules not processed for user `4731e152aa994dffaa524b951c915bab`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:45:24.197538 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-23217d13-bc34-4414-8ece-c49e5947cc6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:45:24.289713 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 993/2010] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:45:24 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:45:24.300824 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1f3f8a-768d-41b8-a65c-9e6a58d38ad4 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:24.302943 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-0f1f3f8a-768d-41b8-a65c-9e6a58d38ad4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:45:24.373379 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1f3f8a-768d-41b8-a65c-9e6a58d38ad4 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:24.377259 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0272a437-a442-4c0a-b76f-841eaf6fbe8b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24d5e2637b7043e08313909d5b06e329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:24.377963 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0272a437-a442-4c0a-b76f-841eaf6fbe8b service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:24.378232 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0272a437-a442-4c0a-b76f-841eaf6fbe8b service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:24.378506 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0272a437-a442-4c0a-b76f-841eaf6fbe8b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:24.442550 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0272a437-a442-4c0a-b76f-841eaf6fbe8b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:24.444146 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0272a437-a442-4c0a-b76f-841eaf6fbe8b service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:24.455872 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1018/2011] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:45:24 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:45:26.117477 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23217d13-bc34-4414-8ece-c49e5947cc6a None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:26.129972 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23217d13-bc34-4414-8ece-c49e5947cc6a None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:26.141596 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af838458-d320-4aa4-b541-b67c9c4fddad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24d5e2637b7043e08313909d5b06e329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:26.142717 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af838458-d320-4aa4-b541-b67c9c4fddad service cinder] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:26.143277 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af838458-d320-4aa4-b541-b67c9c4fddad service cinder] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:26.143769 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af838458-d320-4aa4-b541-b67c9c4fddad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:26.222685 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af838458-d320-4aa4-b541-b67c9c4fddad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:26.223317 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af838458-d320-4aa4-b541-b67c9c4fddad service cinder] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:26.246906 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 994/2012] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:45:26 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:45:26.828662 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3e50e8-f3bd-42a2-8417-3fbdbafa2d6d None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:26.829519 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3e50e8-f3bd-42a2-8417-3fbdbafa2d6d None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:26.829836 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3e50e8-f3bd-42a2-8417-3fbdbafa2d6d None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:26.831376 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1019/2013] 10.222.0.42 () {54 vars in 921 bytes} [Thu Jun 16 17:45:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:45:26.842836 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5bb63a-ad14-4549-a497-cd02305dde49 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:26.843324 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5bb63a-ad14-4549-a497-cd02305dde49 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:26.843675 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5bb63a-ad14-4549-a497-cd02305dde49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:26.899977 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-cd5bb63a-ad14-4549-a497-cd02305dde49 None None] MFA Rules not processed for user `0570d8d9508d4546ae897276f7d181d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:45:26.903281 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-cd5bb63a-ad14-4549-a497-cd02305dde49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:45:26.928890 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3e50e8-f3bd-42a2-8417-3fbdbafa2d6d None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:26.931601 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-ce3e50e8-f3bd-42a2-8417-3fbdbafa2d6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:45:27.017772 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3e50e8-f3bd-42a2-8417-3fbdbafa2d6d None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:27.021512 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:45:27.022621 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ceba22c-458c-4dab-a930-18005492e9a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:27.023509 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceba22c-458c-4dab-a930-18005492e9a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:27.024366 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceba22c-458c-4dab-a930-18005492e9a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:27.024366 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceba22c-458c-4dab-a930-18005492e9a1 admin admin] PATH_INFO: `/v3/users/e577b2f2f18a4be397b6ea6e7f1ba0ad` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:27.025416 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ceba22c-458c-4dab-a930-18005492e9a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e577b2f2f18a4be397b6ea6e7f1ba0ad)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:27.027354 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ceba22c-458c-4dab-a930-18005492e9a1 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:27.036793 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 995/2014] 10.222.0.42 () {58 vars in 1018 bytes} [Thu Jun 16 17:45:26 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:45:27.038111 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5bb63a-ad14-4549-a497-cd02305dde49 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:27.039844 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-cd5bb63a-ad14-4549-a497-cd02305dde49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:45:27.053349 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5bb63a-ad14-4549-a497-cd02305dde49 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:27.057990 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478379a2-5ceb-4788-b4aa-0fa377d6fe40 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24d5e2637b7043e08313909d5b06e329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:27.059377 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478379a2-5ceb-4788-b4aa-0fa377d6fe40 service cinder] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:27.060100 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478379a2-5ceb-4788-b4aa-0fa377d6fe40 service cinder] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:27.060100 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478379a2-5ceb-4788-b4aa-0fa377d6fe40 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:27.073601 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:45:27.075153 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478379a2-5ceb-4788-b4aa-0fa377d6fe40 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:27.077565 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478379a2-5ceb-4788-b4aa-0fa377d6fe40 service cinder] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:27.087323 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-2ceba22c-458c-4dab-a930-18005492e9a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e577b2f2f18a4be397b6ea6e7f1ba0ad'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:27.088255 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 996/2015] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:45:27 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:45:27.088911 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478379a2-5ceb-4788-b4aa-0fa377d6fe40 service cinder] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:27.090252 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-478379a2-5ceb-4788-b4aa-0fa377d6fe40 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:45:27.102742 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-2ceba22c-458c-4dab-a930-18005492e9a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e577b2f2f18a4be397b6ea6e7f1ba0ad'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:27.103955 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-2ceba22c-458c-4dab-a930-18005492e9a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e577b2f2f18a4be397b6ea6e7f1ba0ad'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:27.107453 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478379a2-5ceb-4788-b4aa-0fa377d6fe40 service cinder] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:27.120868 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43c979f-c22d-4df4-be9d-bbd81dac5fb7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0570d8d9508d4546ae897276f7d181d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:27.122665 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43c979f-c22d-4df4-be9d-bbd81dac5fb7 service placement] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:27.123206 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43c979f-c22d-4df4-be9d-bbd81dac5fb7 service placement] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:27.123641 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43c979f-c22d-4df4-be9d-bbd81dac5fb7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:27.125911 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-2ceba22c-458c-4dab-a930-18005492e9a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e577b2f2f18a4be397b6ea6e7f1ba0ad'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:27.126522 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-f43c979f-c22d-4df4-be9d-bbd81dac5fb7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:45:27.142062 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-2ceba22c-458c-4dab-a930-18005492e9a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e577b2f2f18a4be397b6ea6e7f1ba0ad'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:27.148305 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43c979f-c22d-4df4-be9d-bbd81dac5fb7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:27.150164 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1020/2016] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jun 16 17:45:26 2022] DELETE /identity/v3/users/e577b2f2f18a4be397b6ea6e7f1ba0ad => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:45:27.154002 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43c979f-c22d-4df4-be9d-bbd81dac5fb7 service placement] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:27.169624 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 997/2017] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:45:27 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:45:27.342126 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ceba22c-458c-4dab-a930-18005492e9a1 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:27.343602 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-2ceba22c-458c-4dab-a930-18005492e9a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:45:27.418298 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ceba22c-458c-4dab-a930-18005492e9a1 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:27.423359 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5dd13f-501e-4576-b1ee-829f971e28f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:27.424237 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5dd13f-501e-4576-b1ee-829f971e28f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:27.424552 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5dd13f-501e-4576-b1ee-829f971e28f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:27.424848 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5dd13f-501e-4576-b1ee-829f971e28f6 admin admin] PATH_INFO: `/v3/projects/375c9f75db854f22a560c9f6408f16b5` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:27.426034 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5dd13f-501e-4576-b1ee-829f971e28f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=375c9f75db854f22a560c9f6408f16b5)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:27.427950 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5dd13f-501e-4576-b1ee-829f971e28f6 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:27.475789 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-da5dd13f-501e-4576-b1ee-829f971e28f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '375c9f75db854f22a560c9f6408f16b5', 'request_id': 'req-da5dd13f-501e-4576-b1ee-829f971e28f6'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:27.484226 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-da5dd13f-501e-4576-b1ee-829f971e28f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '375c9f75db854f22a560c9f6408f16b5', 'request_id': 'req-da5dd13f-501e-4576-b1ee-829f971e28f6'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:27.491251 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-da5dd13f-501e-4576-b1ee-829f971e28f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '375c9f75db854f22a560c9f6408f16b5', 'request_id': 'req-da5dd13f-501e-4576-b1ee-829f971e28f6'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:27.502932 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-da5dd13f-501e-4576-b1ee-829f971e28f6 admin admin] The token cache is being invalidate because project 375c9f75db854f22a560c9f6408f16b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:45:27.503219 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-da5dd13f-501e-4576-b1ee-829f971e28f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:27.503863 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-da5dd13f-501e-4576-b1ee-829f971e28f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:27.505422 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1021/2018] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jun 16 17:45:27 2022] DELETE /identity/v3/projects/375c9f75db854f22a560c9f6408f16b5 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:45:27.510509 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8793fd08-a14b-41eb-8acf-fa60b3083dc1 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:27.510798 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8793fd08-a14b-41eb-8acf-fa60b3083dc1 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:27.511077 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8793fd08-a14b-41eb-8acf-fa60b3083dc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:27.529512 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-8793fd08-a14b-41eb-8acf-fa60b3083dc1 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:45:27.530608 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-8793fd08-a14b-41eb-8acf-fa60b3083dc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:45:27.592417 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 998/2019] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:45:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:45:27.600297 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499dae06-cfe4-4207-b4c4-e0edfea98d9f None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:27.600619 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499dae06-cfe4-4207-b4c4-e0edfea98d9f None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:27.600899 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499dae06-cfe4-4207-b4c4-e0edfea98d9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:27.618983 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-499dae06-cfe4-4207-b4c4-e0edfea98d9f None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:45:27.620191 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-499dae06-cfe4-4207-b4c4-e0edfea98d9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:45:27.628855 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1022/2020] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:45:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:45:27.633032 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8793fd08-a14b-41eb-8acf-fa60b3083dc1 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:27.643058 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8793fd08-a14b-41eb-8acf-fa60b3083dc1 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:27.647999 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952e7262-5cc0-4bca-a234-5007e83bc6e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:27.648698 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952e7262-5cc0-4bca-a234-5007e83bc6e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:27.648986 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952e7262-5cc0-4bca-a234-5007e83bc6e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:27.649264 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952e7262-5cc0-4bca-a234-5007e83bc6e8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:27.650022 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952e7262-5cc0-4bca-a234-5007e83bc6e8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:45:27.650395 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952e7262-5cc0-4bca-a234-5007e83bc6e8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:27.652098 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952e7262-5cc0-4bca-a234-5007e83bc6e8 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:27.665023 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 999/2021] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jun 16 17:45:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:45:27.669606 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499dae06-cfe4-4207-b4c4-e0edfea98d9f None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:27.682088 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499dae06-cfe4-4207-b4c4-e0edfea98d9f None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:27.691039 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b37b9e-7ac0-49d9-88d5-41a9820fd500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:27.691039 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b37b9e-7ac0-49d9-88d5-41a9820fd500 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:27.691039 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b37b9e-7ac0-49d9-88d5-41a9820fd500 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:27.691039 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b37b9e-7ac0-49d9-88d5-41a9820fd500 admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:27.691436 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b37b9e-7ac0-49d9-88d5-41a9820fd500 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:27.693254 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b37b9e-7ac0-49d9-88d5-41a9820fd500 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:27.719622 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1023/2022] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jun 16 17:45:27 2022] POST /identity/v3/projects => generated 403 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:45:27.723765 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952e7262-5cc0-4bca-a234-5007e83bc6e8 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:27.734013 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952e7262-5cc0-4bca-a234-5007e83bc6e8 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:27.799591 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb7a804-b289-4deb-a34e-3efd14531f7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:27.800184 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb7a804-b289-4deb-a34e-3efd14531f7b admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:27.800508 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb7a804-b289-4deb-a34e-3efd14531f7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:27.800781 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb7a804-b289-4deb-a34e-3efd14531f7b admin admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:27.801478 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb7a804-b289-4deb-a34e-3efd14531f7b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:27.803407 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb7a804-b289-4deb-a34e-3efd14531f7b admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:27.826734 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1000/2023] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jun 16 17:45:27 2022] POST /identity/v3/users => generated 302 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:45:27.831010 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b37b9e-7ac0-49d9-88d5-41a9820fd500 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:27.842140 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b37b9e-7ac0-49d9-88d5-41a9820fd500 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:27.846891 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0735828f-7b68-4c67-afc2-cf71adf9bdfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:27.847554 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0735828f-7b68-4c67-afc2-cf71adf9bdfd admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:27.847842 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0735828f-7b68-4c67-afc2-cf71adf9bdfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:27.848126 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0735828f-7b68-4c67-afc2-cf71adf9bdfd admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:27.848970 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0735828f-7b68-4c67-afc2-cf71adf9bdfd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:27.850490 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0735828f-7b68-4c67-afc2-cf71adf9bdfd admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:27.861617 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1024/2024] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:45:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:45:27.865616 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb7a804-b289-4deb-a34e-3efd14531f7b admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:27.876571 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb7a804-b289-4deb-a34e-3efd14531f7b admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:27.881263 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14077480-a52f-4e50-853a-0737e497b2ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:27.881928 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14077480-a52f-4e50-853a-0737e497b2ca admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:27.882216 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14077480-a52f-4e50-853a-0737e497b2ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:27.882492 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14077480-a52f-4e50-853a-0737e497b2ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:27.883506 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14077480-a52f-4e50-853a-0737e497b2ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:27.885150 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14077480-a52f-4e50-853a-0737e497b2ca admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:27.895572 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1001/2025] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:45:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:45:27.899696 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0735828f-7b68-4c67-afc2-cf71adf9bdfd admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:27.905079 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:45:27.911260 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0735828f-7b68-4c67-afc2-cf71adf9bdfd admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:27.917069 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4982d54a-4af4-4401-9a47-7f31ed5a3116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:27.917950 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4982d54a-4af4-4401-9a47-7f31ed5a3116 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:27.918726 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4982d54a-4af4-4401-9a47-7f31ed5a3116 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:27.919524 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4982d54a-4af4-4401-9a47-7f31ed5a3116 admin admin] PATH_INFO: `/v3/projects/d1f4b50027dc4441997f572710e2096b/users/7adadaca5e71408a8fd5527824cb61af/roles/0be2075946f54c8485f1c275f746478a` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:27.935664 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4982d54a-4af4-4401-9a47-7f31ed5a3116 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1f4b50027dc4441997f572710e2096b, user_id=7adadaca5e71408a8fd5527824cb61af, role_id=0be2075946f54c8485f1c275f746478a)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:27.938356 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4982d54a-4af4-4401-9a47-7f31ed5a3116 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:27.953296 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1025/2026] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:45:27 2022] PUT /identity/v3/projects/d1f4b50027dc4441997f572710e2096b/users/7adadaca5e71408a8fd5527824cb61af/roles/0be2075946f54c8485f1c275f746478a => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:45:27.958418 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d10d961-a2bd-4fe5-975c-da3bbfd1fb70 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:27.958733 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d10d961-a2bd-4fe5-975c-da3bbfd1fb70 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:27.959013 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d10d961-a2bd-4fe5-975c-da3bbfd1fb70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:27.978798 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-1d10d961-a2bd-4fe5-975c-da3bbfd1fb70 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:45:27.980013 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-1d10d961-a2bd-4fe5-975c-da3bbfd1fb70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:45:28.016721 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:45:28.048531 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1002/2027] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:45:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:45:28.060236 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4982d54a-4af4-4401-9a47-7f31ed5a3116 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:28.065164 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-4982d54a-4af4-4401-9a47-7f31ed5a3116 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:45:28.129094 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4982d54a-4af4-4401-9a47-7f31ed5a3116 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:28.132862 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2388e31c-ea82-4b67-b2e9-3721649c4d56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:28.133764 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2388e31c-ea82-4b67-b2e9-3721649c4d56 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:28.134050 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2388e31c-ea82-4b67-b2e9-3721649c4d56 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:28.134327 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2388e31c-ea82-4b67-b2e9-3721649c4d56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:28.144020 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2388e31c-ea82-4b67-b2e9-3721649c4d56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:28.145488 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2388e31c-ea82-4b67-b2e9-3721649c4d56 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:28.155712 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1026/2028] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:45:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 17:45:28.851091 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281d30d6-5abe-4072-9735-16f27c7fc366 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:28.851569 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281d30d6-5abe-4072-9735-16f27c7fc366 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:28.851683 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281d30d6-5abe-4072-9735-16f27c7fc366 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:28.890361 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-281d30d6-5abe-4072-9735-16f27c7fc366 None None] MFA Rules not processed for user `7adadaca5e71408a8fd5527824cb61af`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:45:28.891633 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-281d30d6-5abe-4072-9735-16f27c7fc366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:45:28.961074 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1003/2029] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:45:28 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:45:28.966036 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2388e31c-ea82-4b67-b2e9-3721649c4d56 service neutron] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:28.976421 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2388e31c-ea82-4b67-b2e9-3721649c4d56 service neutron] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:28.982697 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbb7a27-6324-44ce-bcae-06b188876400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:28.983411 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbb7a27-6324-44ce-bcae-06b188876400 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:28.983632 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbb7a27-6324-44ce-bcae-06b188876400 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:28.983978 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbb7a27-6324-44ce-bcae-06b188876400 admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:28.984728 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbb7a27-6324-44ce-bcae-06b188876400 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:28.986627 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbb7a27-6324-44ce-bcae-06b188876400 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:29.019378 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1027/2030] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jun 16 17:45:28 2022] POST /identity/v3/projects => generated 405 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:45:29.023380 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281d30d6-5abe-4072-9735-16f27c7fc366 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:29.035044 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281d30d6-5abe-4072-9735-16f27c7fc366 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:29.096003 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c58fe6d-7aa6-4771-8d75-025f5da957d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:29.096636 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c58fe6d-7aa6-4771-8d75-025f5da957d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:29.096920 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c58fe6d-7aa6-4771-8d75-025f5da957d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:29.097201 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c58fe6d-7aa6-4771-8d75-025f5da957d4 admin admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:29.097895 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c58fe6d-7aa6-4771-8d75-025f5da957d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:29.099818 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c58fe6d-7aa6-4771-8d75-025f5da957d4 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:29.120488 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1004/2031] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jun 16 17:45:29 2022] POST /identity/v3/users => generated 309 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:45:29.125547 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbb7a27-6324-44ce-bcae-06b188876400 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:29.136257 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbb7a27-6324-44ce-bcae-06b188876400 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:29.142061 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd73d1bc-756b-4afc-8e92-e0541f3fa9e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:29.142720 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd73d1bc-756b-4afc-8e92-e0541f3fa9e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:29.143005 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd73d1bc-756b-4afc-8e92-e0541f3fa9e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:29.143284 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd73d1bc-756b-4afc-8e92-e0541f3fa9e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:29.144003 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd73d1bc-756b-4afc-8e92-e0541f3fa9e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:29.145630 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd73d1bc-756b-4afc-8e92-e0541f3fa9e6 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:29.156924 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1028/2032] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:45:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:45:29.161216 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c58fe6d-7aa6-4771-8d75-025f5da957d4 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:29.171307 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c58fe6d-7aa6-4771-8d75-025f5da957d4 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:29.175949 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd68b7b-5eab-46c2-95b0-ce7ecae09582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:29.176843 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd68b7b-5eab-46c2-95b0-ce7ecae09582 admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:29.177148 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd68b7b-5eab-46c2-95b0-ce7ecae09582 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:29.177419 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd68b7b-5eab-46c2-95b0-ce7ecae09582 admin admin] PATH_INFO: `/v3/projects/8e99b38b776d49df88853e277d434679/users/908b34234e30497fb7a2acd2f3d533f1/roles/3ca2dcbbd29d4a31a3ab78f2299bc239` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:29.192625 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd68b7b-5eab-46c2-95b0-ce7ecae09582 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e99b38b776d49df88853e277d434679, user_id=908b34234e30497fb7a2acd2f3d533f1, role_id=3ca2dcbbd29d4a31a3ab78f2299bc239)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:29.195244 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd68b7b-5eab-46c2-95b0-ce7ecae09582 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:29.209243 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1005/2033] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:45:29 2022] PUT /identity/v3/projects/8e99b38b776d49df88853e277d434679/users/908b34234e30497fb7a2acd2f3d533f1/roles/3ca2dcbbd29d4a31a3ab78f2299bc239 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:45:29.213253 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd73d1bc-756b-4afc-8e92-e0541f3fa9e6 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:29.224064 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd73d1bc-756b-4afc-8e92-e0541f3fa9e6 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:29.282109 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9345e2a5-2296-4dc4-b905-76ffb4f08d7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:29.282793 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9345e2a5-2296-4dc4-b905-76ffb4f08d7f admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:29.283073 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9345e2a5-2296-4dc4-b905-76ffb4f08d7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:29.283345 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9345e2a5-2296-4dc4-b905-76ffb4f08d7f admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:29.284057 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9345e2a5-2296-4dc4-b905-76ffb4f08d7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:29.285684 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9345e2a5-2296-4dc4-b905-76ffb4f08d7f admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:29.295836 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1029/2034] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:45:29 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:45:29.300013 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd68b7b-5eab-46c2-95b0-ce7ecae09582 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:29.311173 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd68b7b-5eab-46c2-95b0-ce7ecae09582 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:29.315741 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea82f0dd-868b-4d79-983f-7f70e0322c7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:29.316666 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea82f0dd-868b-4d79-983f-7f70e0322c7e admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:29.316953 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea82f0dd-868b-4d79-983f-7f70e0322c7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:29.317260 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea82f0dd-868b-4d79-983f-7f70e0322c7e admin admin] PATH_INFO: `/v3/projects/8e99b38b776d49df88853e277d434679/users/908b34234e30497fb7a2acd2f3d533f1/roles/3ca2dcbbd29d4a31a3ab78f2299bc239` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:29.319214 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea82f0dd-868b-4d79-983f-7f70e0322c7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e99b38b776d49df88853e277d434679, user_id=908b34234e30497fb7a2acd2f3d533f1, role_id=3ca2dcbbd29d4a31a3ab78f2299bc239)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:29.321876 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea82f0dd-868b-4d79-983f-7f70e0322c7e admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:29.332631 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1006/2035] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:45:29 2022] PUT /identity/v3/projects/8e99b38b776d49df88853e277d434679/users/908b34234e30497fb7a2acd2f3d533f1/roles/3ca2dcbbd29d4a31a3ab78f2299bc239 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:45:29.990263 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e17790f-d852-4d70-a0aa-6666963ee38e None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:29.991295 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e17790f-d852-4d70-a0aa-6666963ee38e None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:29.991718 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e17790f-d852-4d70-a0aa-6666963ee38e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:30.023883 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-7e17790f-d852-4d70-a0aa-6666963ee38e None None] MFA Rules not processed for user `908b34234e30497fb7a2acd2f3d533f1`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:45:30.025400 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-7e17790f-d852-4d70-a0aa-6666963ee38e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:45:30.068804 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:45:30.116567 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1030/2036] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:45:29 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:45:30.640093 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77ff7bd-c8c7-4903-985b-471e370e2e12 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:30.640645 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77ff7bd-c8c7-4903-985b-471e370e2e12 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:30.640645 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77ff7bd-c8c7-4903-985b-471e370e2e12 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:30.644067 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1007/2037] 10.222.0.42 () {54 vars in 921 bytes} [Thu Jun 16 17:45:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:45:30.653489 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba05551-4f99-48e5-ad6c-f7b49e0d7425 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:30.653893 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba05551-4f99-48e5-ad6c-f7b49e0d7425 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:30.654198 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba05551-4f99-48e5-ad6c-f7b49e0d7425 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:30.678137 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-dba05551-4f99-48e5-ad6c-f7b49e0d7425 None None] MFA Rules not processed for user `0570d8d9508d4546ae897276f7d181d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:45:30.679507 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-dba05551-4f99-48e5-ad6c-f7b49e0d7425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:45:30.771248 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1031/2038] 10.222.0.42 () {58 vars in 1018 bytes} [Thu Jun 16 17:45:30 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:45:30.788949 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77ff7bd-c8c7-4903-985b-471e370e2e12 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:30.791043 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-f77ff7bd-c8c7-4903-985b-471e370e2e12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:45:30.811755 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77ff7bd-c8c7-4903-985b-471e370e2e12 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:30.818101 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0809e2-3ba6-4da6-93e4-1a1d06c3ab3c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0570d8d9508d4546ae897276f7d181d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:30.818754 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0809e2-3ba6-4da6-93e4-1a1d06c3ab3c service placement] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:30.819031 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0809e2-3ba6-4da6-93e4-1a1d06c3ab3c service placement] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:30.819304 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0809e2-3ba6-4da6-93e4-1a1d06c3ab3c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:30.829331 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0809e2-3ba6-4da6-93e4-1a1d06c3ab3c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:30.839546 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0809e2-3ba6-4da6-93e4-1a1d06c3ab3c service placement] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:30.855157 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1008/2039] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:45:30 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:45:31.116903 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba05551-4f99-48e5-ad6c-f7b49e0d7425 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:31.118348 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-dba05551-4f99-48e5-ad6c-f7b49e0d7425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:45:31.218779 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba05551-4f99-48e5-ad6c-f7b49e0d7425 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:31.228139 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1336c922-2b53-4be9-af71-13b6508c5511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:31.228139 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1336c922-2b53-4be9-af71-13b6508c5511 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:31.228139 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1336c922-2b53-4be9-af71-13b6508c5511 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:31.228139 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1336c922-2b53-4be9-af71-13b6508c5511 admin admin] PATH_INFO: `/v3/users/e78c9b1a1c7943509f234a7d1d6a6767` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:31.238856 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1336c922-2b53-4be9-af71-13b6508c5511 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e78c9b1a1c7943509f234a7d1d6a6767)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:31.241128 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1336c922-2b53-4be9-af71-13b6508c5511 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:31.285741 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0809e2-3ba6-4da6-93e4-1a1d06c3ab3c service placement] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:31.294920 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-1336c922-2b53-4be9-af71-13b6508c5511 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e78c9b1a1c7943509f234a7d1d6a6767'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:31.298543 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0809e2-3ba6-4da6-93e4-1a1d06c3ab3c service placement] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:31.303119 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9224bb27-5a09-4a82-b8aa-c72a753d60ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:31.303991 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9224bb27-5a09-4a82-b8aa-c72a753d60ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:31.304264 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9224bb27-5a09-4a82-b8aa-c72a753d60ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:31.304584 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9224bb27-5a09-4a82-b8aa-c72a753d60ce admin admin] PATH_INFO: `/v3/users/7adadaca5e71408a8fd5527824cb61af` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:31.306121 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9224bb27-5a09-4a82-b8aa-c72a753d60ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=7adadaca5e71408a8fd5527824cb61af)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:31.307752 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9224bb27-5a09-4a82-b8aa-c72a753d60ce admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:31.311312 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-1336c922-2b53-4be9-af71-13b6508c5511 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e78c9b1a1c7943509f234a7d1d6a6767'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:31.311515 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:45:31.312023 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-1336c922-2b53-4be9-af71-13b6508c5511 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e78c9b1a1c7943509f234a7d1d6a6767'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:31.314266 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:45:31.322222 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-1336c922-2b53-4be9-af71-13b6508c5511 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e78c9b1a1c7943509f234a7d1d6a6767'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:31.329703 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-1336c922-2b53-4be9-af71-13b6508c5511 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e78c9b1a1c7943509f234a7d1d6a6767'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:31.337000 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1032/2040] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jun 16 17:45:31 2022] DELETE /identity/v3/users/e78c9b1a1c7943509f234a7d1d6a6767 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:45:31.341760 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe16c408-206d-471d-9254-63f7d27ace8c None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:31.342065 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe16c408-206d-471d-9254-63f7d27ace8c None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:31.342342 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe16c408-206d-471d-9254-63f7d27ace8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:31.360150 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-fe16c408-206d-471d-9254-63f7d27ace8c None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:45:31.361403 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-fe16c408-206d-471d-9254-63f7d27ace8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:45:31.361758 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-9224bb27-5a09-4a82-b8aa-c72a753d60ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7adadaca5e71408a8fd5527824cb61af'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:31.362532 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-9224bb27-5a09-4a82-b8aa-c72a753d60ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7adadaca5e71408a8fd5527824cb61af'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:31.370580 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-9224bb27-5a09-4a82-b8aa-c72a753d60ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7adadaca5e71408a8fd5527824cb61af'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:31.372084 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-9224bb27-5a09-4a82-b8aa-c72a753d60ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7adadaca5e71408a8fd5527824cb61af'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:31.385428 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-9224bb27-5a09-4a82-b8aa-c72a753d60ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7adadaca5e71408a8fd5527824cb61af'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:31.398829 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1009/2041] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jun 16 17:45:31 2022] DELETE /identity/v3/users/7adadaca5e71408a8fd5527824cb61af => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:45:31.428831 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1033/2042] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:45:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:45:31.436247 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9224bb27-5a09-4a82-b8aa-c72a753d60ce admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:31.438559 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-9224bb27-5a09-4a82-b8aa-c72a753d60ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:45:31.506909 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9224bb27-5a09-4a82-b8aa-c72a753d60ce admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:31.511727 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f09d81-ea7c-4d22-8688-a5f7f6c0f53f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:31.513193 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f09d81-ea7c-4d22-8688-a5f7f6c0f53f service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:31.513483 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f09d81-ea7c-4d22-8688-a5f7f6c0f53f service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:31.513760 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f09d81-ea7c-4d22-8688-a5f7f6c0f53f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:31.591458 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe16c408-206d-471d-9254-63f7d27ace8c None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:31.592998 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-fe16c408-206d-471d-9254-63f7d27ace8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:45:31.596227 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f09d81-ea7c-4d22-8688-a5f7f6c0f53f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:31.597791 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f09d81-ea7c-4d22-8688-a5f7f6c0f53f service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:31.608839 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1010/2043] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:45:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:45:31.635651 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe16c408-206d-471d-9254-63f7d27ace8c None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:31.649637 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9451b7-323f-4005-b65f-5f0d57b5426f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:31.650582 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9451b7-323f-4005-b65f-5f0d57b5426f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:31.651151 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9451b7-323f-4005-b65f-5f0d57b5426f admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:31.651311 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9451b7-323f-4005-b65f-5f0d57b5426f admin admin] PATH_INFO: `/v3/projects/d1f4b50027dc4441997f572710e2096b` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:31.652746 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9451b7-323f-4005-b65f-5f0d57b5426f admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1f4b50027dc4441997f572710e2096b)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:31.655305 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9451b7-323f-4005-b65f-5f0d57b5426f admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:31.711805 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-1f9451b7-323f-4005-b65f-5f0d57b5426f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1f4b50027dc4441997f572710e2096b', 'request_id': 'req-1f9451b7-323f-4005-b65f-5f0d57b5426f'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:31.720680 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-1f9451b7-323f-4005-b65f-5f0d57b5426f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1f4b50027dc4441997f572710e2096b', 'request_id': 'req-1f9451b7-323f-4005-b65f-5f0d57b5426f'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:31.727323 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-1f9451b7-323f-4005-b65f-5f0d57b5426f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1f4b50027dc4441997f572710e2096b', 'request_id': 'req-1f9451b7-323f-4005-b65f-5f0d57b5426f'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:31.739164 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-1f9451b7-323f-4005-b65f-5f0d57b5426f admin admin] The token cache is being invalidate because project d1f4b50027dc4441997f572710e2096b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:45:31.739459 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-1f9451b7-323f-4005-b65f-5f0d57b5426f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:31.740134 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-1f9451b7-323f-4005-b65f-5f0d57b5426f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:31.741693 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1034/2044] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jun 16 17:45:31 2022] DELETE /identity/v3/projects/d1f4b50027dc4441997f572710e2096b => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:45:31.745904 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f09d81-ea7c-4d22-8688-a5f7f6c0f53f service neutron] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:31.756508 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f09d81-ea7c-4d22-8688-a5f7f6c0f53f service neutron] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:31.790345 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:45:31.826650 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849de6a6-87dd-4d85-95ae-66d6c343552c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:31.827560 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849de6a6-87dd-4d85-95ae-66d6c343552c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:31.827845 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849de6a6-87dd-4d85-95ae-66d6c343552c admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:31.828272 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849de6a6-87dd-4d85-95ae-66d6c343552c admin admin] PATH_INFO: `/v3/users/908b34234e30497fb7a2acd2f3d533f1` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:31.829573 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849de6a6-87dd-4d85-95ae-66d6c343552c admin admin] RBAC: Authorizing `identity:delete_user(user_id=908b34234e30497fb7a2acd2f3d533f1)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:31.831536 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849de6a6-87dd-4d85-95ae-66d6c343552c admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:31.883082 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-849de6a6-87dd-4d85-95ae-66d6c343552c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '908b34234e30497fb7a2acd2f3d533f1'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:31.884187 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-849de6a6-87dd-4d85-95ae-66d6c343552c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '908b34234e30497fb7a2acd2f3d533f1'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:31.894877 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-849de6a6-87dd-4d85-95ae-66d6c343552c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '908b34234e30497fb7a2acd2f3d533f1'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:31.895617 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-849de6a6-87dd-4d85-95ae-66d6c343552c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '908b34234e30497fb7a2acd2f3d533f1'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:31.908285 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-849de6a6-87dd-4d85-95ae-66d6c343552c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '908b34234e30497fb7a2acd2f3d533f1'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:31.920533 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1011/2045] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jun 16 17:45:31 2022] DELETE /identity/v3/users/908b34234e30497fb7a2acd2f3d533f1 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:45:31.984059 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9451b7-323f-4005-b65f-5f0d57b5426f admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:31.986774 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-1f9451b7-323f-4005-b65f-5f0d57b5426f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:45:32.011440 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9451b7-323f-4005-b65f-5f0d57b5426f admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:32.090067 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ad5546-9f66-433c-872f-0e3adffe7774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:32.091155 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ad5546-9f66-433c-872f-0e3adffe7774 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:32.091442 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ad5546-9f66-433c-872f-0e3adffe7774 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:32.091774 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ad5546-9f66-433c-872f-0e3adffe7774 admin admin] PATH_INFO: `/v3/projects/bed678c9985d4f249517d30769e64f1d` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:32.093134 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ad5546-9f66-433c-872f-0e3adffe7774 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bed678c9985d4f249517d30769e64f1d)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:32.095050 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ad5546-9f66-433c-872f-0e3adffe7774 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:32.106172 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:45:32.126671 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849de6a6-87dd-4d85-95ae-66d6c343552c admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:32.128508 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-849de6a6-87dd-4d85-95ae-66d6c343552c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:45:32.150381 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849de6a6-87dd-4d85-95ae-66d6c343552c admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:32.157199 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-d5ad5546-9f66-433c-872f-0e3adffe7774 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bed678c9985d4f249517d30769e64f1d', 'request_id': 'req-d5ad5546-9f66-433c-872f-0e3adffe7774'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:32.167779 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-d5ad5546-9f66-433c-872f-0e3adffe7774 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bed678c9985d4f249517d30769e64f1d', 'request_id': 'req-d5ad5546-9f66-433c-872f-0e3adffe7774'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:32.175717 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-d5ad5546-9f66-433c-872f-0e3adffe7774 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bed678c9985d4f249517d30769e64f1d', 'request_id': 'req-d5ad5546-9f66-433c-872f-0e3adffe7774'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:32.188149 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-d5ad5546-9f66-433c-872f-0e3adffe7774 admin admin] The token cache is being invalidate because project bed678c9985d4f249517d30769e64f1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:45:32.188478 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-d5ad5546-9f66-433c-872f-0e3adffe7774 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:32.189156 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-d5ad5546-9f66-433c-872f-0e3adffe7774 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:32.190997 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1035/2046] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jun 16 17:45:31 2022] DELETE /identity/v3/projects/bed678c9985d4f249517d30769e64f1d => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:45:32.194679 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ad5546-9f66-433c-872f-0e3adffe7774 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:32.196054 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-d5ad5546-9f66-433c-872f-0e3adffe7774 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:45:32.227428 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d916b91f-0293-46c9-a1f2-8f284ae078c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:32.228245 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d916b91f-0293-46c9-a1f2-8f284ae078c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:32.228563 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d916b91f-0293-46c9-a1f2-8f284ae078c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:32.228844 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d916b91f-0293-46c9-a1f2-8f284ae078c4 admin admin] PATH_INFO: `/v3/projects/8e99b38b776d49df88853e277d434679` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:32.230064 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d916b91f-0293-46c9-a1f2-8f284ae078c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e99b38b776d49df88853e277d434679)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:32.232118 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d916b91f-0293-46c9-a1f2-8f284ae078c4 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:32.272801 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ad5546-9f66-433c-872f-0e3adffe7774 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:32.279393 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e71df14-59dd-4675-a1cf-c67e75bd2563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:32.280772 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e71df14-59dd-4675-a1cf-c67e75bd2563 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:32.281243 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e71df14-59dd-4675-a1cf-c67e75bd2563 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:32.281649 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e71df14-59dd-4675-a1cf-c67e75bd2563 admin admin] PATH_INFO: `/v3/users/4b456bff3dd74f8d868ba9c2697ab086` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:32.283719 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e71df14-59dd-4675-a1cf-c67e75bd2563 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b456bff3dd74f8d868ba9c2697ab086)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:32.286841 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e71df14-59dd-4675-a1cf-c67e75bd2563 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:32.299088 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-d916b91f-0293-46c9-a1f2-8f284ae078c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e99b38b776d49df88853e277d434679', 'request_id': 'req-d916b91f-0293-46c9-a1f2-8f284ae078c4'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:32.303387 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:45:32.308143 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-d916b91f-0293-46c9-a1f2-8f284ae078c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e99b38b776d49df88853e277d434679', 'request_id': 'req-d916b91f-0293-46c9-a1f2-8f284ae078c4'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:32.316383 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-d916b91f-0293-46c9-a1f2-8f284ae078c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e99b38b776d49df88853e277d434679', 'request_id': 'req-d916b91f-0293-46c9-a1f2-8f284ae078c4'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:32.328229 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-d916b91f-0293-46c9-a1f2-8f284ae078c4 admin admin] The token cache is being invalidate because project 8e99b38b776d49df88853e277d434679 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80357) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:45:32.328549 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-d916b91f-0293-46c9-a1f2-8f284ae078c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:32.329200 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-d916b91f-0293-46c9-a1f2-8f284ae078c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:32.330749 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1012/2047] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jun 16 17:45:32 2022] DELETE /identity/v3/projects/8e99b38b776d49df88853e277d434679 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:45:32.335544 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b974f792-ee6d-4c5e-9271-66b3110820f7 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:32.335838 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b974f792-ee6d-4c5e-9271-66b3110820f7 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:32.336128 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b974f792-ee6d-4c5e-9271-66b3110820f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:32.343202 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-3e71df14-59dd-4675-a1cf-c67e75bd2563 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b456bff3dd74f8d868ba9c2697ab086'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:32.354495 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-b974f792-ee6d-4c5e-9271-66b3110820f7 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:45:32.356250 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-b974f792-ee6d-4c5e-9271-66b3110820f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:45:32.356475 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-3e71df14-59dd-4675-a1cf-c67e75bd2563 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b456bff3dd74f8d868ba9c2697ab086'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:32.357338 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-3e71df14-59dd-4675-a1cf-c67e75bd2563 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b456bff3dd74f8d868ba9c2697ab086'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:32.364791 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-3e71df14-59dd-4675-a1cf-c67e75bd2563 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b456bff3dd74f8d868ba9c2697ab086'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:32.373569 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-3e71df14-59dd-4675-a1cf-c67e75bd2563 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b456bff3dd74f8d868ba9c2697ab086'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:32.379728 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1036/2048] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jun 16 17:45:32 2022] DELETE /identity/v3/users/4b456bff3dd74f8d868ba9c2697ab086 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:45:32.427765 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1013/2049] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:45:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:45:32.438097 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6cfbbe-78c3-4d37-b259-d6f2cec81347 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:32.438389 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6cfbbe-78c3-4d37-b259-d6f2cec81347 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:32.438840 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6cfbbe-78c3-4d37-b259-d6f2cec81347 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:32.458444 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-6c6cfbbe-78c3-4d37-b259-d6f2cec81347 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:45:32.459668 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-6c6cfbbe-78c3-4d37-b259-d6f2cec81347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:45:32.473496 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:45:32.526384 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1037/2050] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:45:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:45:32.530768 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b974f792-ee6d-4c5e-9271-66b3110820f7 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:32.542774 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b974f792-ee6d-4c5e-9271-66b3110820f7 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:32.548463 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae64049-2031-4f76-a83b-c19c5615c135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:32.549132 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae64049-2031-4f76-a83b-c19c5615c135 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:32.549417 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae64049-2031-4f76-a83b-c19c5615c135 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:32.549699 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae64049-2031-4f76-a83b-c19c5615c135 admin admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:32.550485 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae64049-2031-4f76-a83b-c19c5615c135 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:45:32.550798 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae64049-2031-4f76-a83b-c19c5615c135 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:32.552473 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae64049-2031-4f76-a83b-c19c5615c135 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:32.567389 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1014/2051] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jun 16 17:45:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:45:32.571492 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6cfbbe-78c3-4d37-b259-d6f2cec81347 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:32.581611 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6cfbbe-78c3-4d37-b259-d6f2cec81347 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:32.586514 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95053af7-1ba9-4b9f-99f0-15b8d9cfc286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:32.587170 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95053af7-1ba9-4b9f-99f0-15b8d9cfc286 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:32.587448 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95053af7-1ba9-4b9f-99f0-15b8d9cfc286 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:32.587720 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95053af7-1ba9-4b9f-99f0-15b8d9cfc286 admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:32.588511 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95053af7-1ba9-4b9f-99f0-15b8d9cfc286 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:32.590696 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95053af7-1ba9-4b9f-99f0-15b8d9cfc286 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:32.618515 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1038/2052] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jun 16 17:45:32 2022] POST /identity/v3/projects => generated 381 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:45:32.623031 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae64049-2031-4f76-a83b-c19c5615c135 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:32.633707 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae64049-2031-4f76-a83b-c19c5615c135 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:32.680591 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95053af7-1ba9-4b9f-99f0-15b8d9cfc286 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:32.683002 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-95053af7-1ba9-4b9f-99f0-15b8d9cfc286 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:45:32.704367 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92b15f5-6295-4a65-86fe-f3b83f2a9e9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:32.705591 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92b15f5-6295-4a65-86fe-f3b83f2a9e9f admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:32.706037 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92b15f5-6295-4a65-86fe-f3b83f2a9e9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:32.706386 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92b15f5-6295-4a65-86fe-f3b83f2a9e9f admin admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:32.707095 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92b15f5-6295-4a65-86fe-f3b83f2a9e9f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:32.709069 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92b15f5-6295-4a65-86fe-f3b83f2a9e9f admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:32.730301 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1015/2053] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jun 16 17:45:32 2022] POST /identity/v3/users => generated 291 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:45:32.734911 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92b15f5-6295-4a65-86fe-f3b83f2a9e9f admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:32.744875 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92b15f5-6295-4a65-86fe-f3b83f2a9e9f admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:32.749730 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac08e69c-1563-4ceb-8f43-d8254f7cb9e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:32.750398 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac08e69c-1563-4ceb-8f43-d8254f7cb9e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:32.750683 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac08e69c-1563-4ceb-8f43-d8254f7cb9e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:32.750964 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac08e69c-1563-4ceb-8f43-d8254f7cb9e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:32.751690 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac08e69c-1563-4ceb-8f43-d8254f7cb9e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:32.753395 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac08e69c-1563-4ceb-8f43-d8254f7cb9e5 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:32.763278 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95053af7-1ba9-4b9f-99f0-15b8d9cfc286 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:32.765482 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1016/2054] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:45:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 17:45:32.768444 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b70a12-4650-47b0-852d-ade3ea3146e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:32.769325 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b70a12-4650-47b0-852d-ade3ea3146e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:32.769615 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac08e69c-1563-4ceb-8f43-d8254f7cb9e5 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:32.769861 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b70a12-4650-47b0-852d-ade3ea3146e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:32.769981 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b70a12-4650-47b0-852d-ade3ea3146e5 admin admin] PATH_INFO: `/v3/projects/e4bb49572d1f41b6962bf7d1b052bf91` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:32.771126 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b70a12-4650-47b0-852d-ade3ea3146e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4bb49572d1f41b6962bf7d1b052bf91)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:32.773140 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b70a12-4650-47b0-852d-ade3ea3146e5 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:32.780467 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac08e69c-1563-4ceb-8f43-d8254f7cb9e5 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:32.785211 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a4066a-72cb-4b61-93f9-1e592cd8b9ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:32.785864 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a4066a-72cb-4b61-93f9-1e592cd8b9ee admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:32.786148 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a4066a-72cb-4b61-93f9-1e592cd8b9ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:32.786423 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a4066a-72cb-4b61-93f9-1e592cd8b9ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:32.787146 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a4066a-72cb-4b61-93f9-1e592cd8b9ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:32.788775 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a4066a-72cb-4b61-93f9-1e592cd8b9ee admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:32.799514 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1017/2055] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:45:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 17:45:32.803493 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a4066a-72cb-4b61-93f9-1e592cd8b9ee admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:32.813778 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a4066a-72cb-4b61-93f9-1e592cd8b9ee admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:32.826689 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-19b70a12-4650-47b0-852d-ade3ea3146e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4bb49572d1f41b6962bf7d1b052bf91', 'request_id': 'req-19b70a12-4650-47b0-852d-ade3ea3146e5'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:32.835908 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-19b70a12-4650-47b0-852d-ade3ea3146e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4bb49572d1f41b6962bf7d1b052bf91', 'request_id': 'req-19b70a12-4650-47b0-852d-ade3ea3146e5'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:32.843189 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-19b70a12-4650-47b0-852d-ade3ea3146e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4bb49572d1f41b6962bf7d1b052bf91', 'request_id': 'req-19b70a12-4650-47b0-852d-ade3ea3146e5'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:32.847323 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:45:32.854522 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-19b70a12-4650-47b0-852d-ade3ea3146e5 admin admin] The token cache is being invalidate because project e4bb49572d1f41b6962bf7d1b052bf91 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:45:32.854813 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-19b70a12-4650-47b0-852d-ade3ea3146e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:32.855474 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-19b70a12-4650-47b0-852d-ade3ea3146e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:32.857484 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1039/2056] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jun 16 17:45:32 2022] DELETE /identity/v3/projects/e4bb49572d1f41b6962bf7d1b052bf91 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:45:32.861390 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b70a12-4650-47b0-852d-ade3ea3146e5 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:32.863053 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-19b70a12-4650-47b0-852d-ade3ea3146e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:45:32.879845 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a2bb98-9528-4fc2-bd19-7e050c3c2795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:32.881009 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a2bb98-9528-4fc2-bd19-7e050c3c2795 admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:32.881293 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a2bb98-9528-4fc2-bd19-7e050c3c2795 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:32.881567 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a2bb98-9528-4fc2-bd19-7e050c3c2795 admin admin] PATH_INFO: `/v3/projects/1f044705a816445da5ede323dff39282/users/23bbe5f1b48b4d3cb142a43bf7038df7/roles/0be2075946f54c8485f1c275f746478a` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:32.897177 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a2bb98-9528-4fc2-bd19-7e050c3c2795 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f044705a816445da5ede323dff39282, user_id=23bbe5f1b48b4d3cb142a43bf7038df7, role_id=0be2075946f54c8485f1c275f746478a)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:32.897952 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:45:32.899848 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a2bb98-9528-4fc2-bd19-7e050c3c2795 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:32.915330 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1018/2057] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:45:32 2022] PUT /identity/v3/projects/1f044705a816445da5ede323dff39282/users/23bbe5f1b48b4d3cb142a43bf7038df7/roles/0be2075946f54c8485f1c275f746478a => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:45:32.927434 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b86444-f2b3-416a-8af7-a7f9a9dd9483 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:32.927732 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b86444-f2b3-416a-8af7-a7f9a9dd9483 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:32.928020 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b86444-f2b3-416a-8af7-a7f9a9dd9483 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:32.941697 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b70a12-4650-47b0-852d-ade3ea3146e5 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:32.955591 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-68b86444-f2b3-416a-8af7-a7f9a9dd9483 None None] MFA Rules not processed for user `23bbe5f1b48b4d3cb142a43bf7038df7`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:45:32.957014 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-68b86444-f2b3-416a-8af7-a7f9a9dd9483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:45:33.002666 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1183262b-196e-4cc0-8655-d099e8ab80c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:33.004764 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1183262b-196e-4cc0-8655-d099e8ab80c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:33.005060 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1183262b-196e-4cc0-8655-d099e8ab80c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:33.005324 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1183262b-196e-4cc0-8655-d099e8ab80c6 admin admin] PATH_INFO: `/v3/users/4731e152aa994dffaa524b951c915bab` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:33.006654 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1183262b-196e-4cc0-8655-d099e8ab80c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4731e152aa994dffaa524b951c915bab)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:33.008986 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1183262b-196e-4cc0-8655-d099e8ab80c6 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:33.031377 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1019/2058] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:45:32 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:45:33.036057 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b86444-f2b3-416a-8af7-a7f9a9dd9483 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:33.037610 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-68b86444-f2b3-416a-8af7-a7f9a9dd9483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:45:33.053730 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b86444-f2b3-416a-8af7-a7f9a9dd9483 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:33.058555 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8770c027-42ce-456e-a515-03b884f2ac3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:33.059903 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8770c027-42ce-456e-a515-03b884f2ac3c admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:33.060186 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8770c027-42ce-456e-a515-03b884f2ac3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:33.060547 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8770c027-42ce-456e-a515-03b884f2ac3c admin admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:33.061242 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8770c027-42ce-456e-a515-03b884f2ac3c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:33.063314 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8770c027-42ce-456e-a515-03b884f2ac3c admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:33.066540 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-1183262b-196e-4cc0-8655-d099e8ab80c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4731e152aa994dffaa524b951c915bab'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:33.082507 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-1183262b-196e-4cc0-8655-d099e8ab80c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4731e152aa994dffaa524b951c915bab'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:33.083357 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-1183262b-196e-4cc0-8655-d099e8ab80c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4731e152aa994dffaa524b951c915bab'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:33.093813 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1020/2059] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jun 16 17:45:33 2022] POST /identity/v3/projects => generated 381 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:45:33.094799 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-1183262b-196e-4cc0-8655-d099e8ab80c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4731e152aa994dffaa524b951c915bab'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:33.098061 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8770c027-42ce-456e-a515-03b884f2ac3c admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:33.099828 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-8770c027-42ce-456e-a515-03b884f2ac3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:45:33.105153 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-1183262b-196e-4cc0-8655-d099e8ab80c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4731e152aa994dffaa524b951c915bab'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:33.119317 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1040/2060] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jun 16 17:45:32 2022] DELETE /identity/v3/users/4731e152aa994dffaa524b951c915bab => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:45:33.191693 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8770c027-42ce-456e-a515-03b884f2ac3c admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:33.226381 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:45:33.284184 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a1654c-ecf3-4347-9a1e-f5944faee393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:33.284917 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a1654c-ecf3-4347-9a1e-f5944faee393 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:33.285223 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a1654c-ecf3-4347-9a1e-f5944faee393 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:33.285485 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a1654c-ecf3-4347-9a1e-f5944faee393 admin admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:33.286211 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a1654c-ecf3-4347-9a1e-f5944faee393 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:33.288193 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a1654c-ecf3-4347-9a1e-f5944faee393 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:33.314683 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1021/2061] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jun 16 17:45:33 2022] POST /identity/v3/users => generated 297 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:45:33.318230 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1183262b-196e-4cc0-8655-d099e8ab80c6 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:33.329348 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1183262b-196e-4cc0-8655-d099e8ab80c6 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:33.334092 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0726201-c78a-4083-9c77-7f6371b4035f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:33.334760 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0726201-c78a-4083-9c77-7f6371b4035f admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:33.335037 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0726201-c78a-4083-9c77-7f6371b4035f admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:33.335322 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0726201-c78a-4083-9c77-7f6371b4035f admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:33.336042 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0726201-c78a-4083-9c77-7f6371b4035f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:33.337712 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0726201-c78a-4083-9c77-7f6371b4035f admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:33.351586 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1041/2062] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:45:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:45:33.352674 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a1654c-ecf3-4347-9a1e-f5944faee393 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:33.363816 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a1654c-ecf3-4347-9a1e-f5944faee393 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:33.368827 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fba9bd-506a-4efe-b0eb-8a79c6093817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:33.369903 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fba9bd-506a-4efe-b0eb-8a79c6093817 admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:33.370278 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fba9bd-506a-4efe-b0eb-8a79c6093817 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:33.370559 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fba9bd-506a-4efe-b0eb-8a79c6093817 admin admin] PATH_INFO: `/v3/projects/4b095b322f364d7c9f85d62a60d959ba/users/c3c91ce49691467a8aa79eb249c90277/roles/3ca2dcbbd29d4a31a3ab78f2299bc239` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:33.387959 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fba9bd-506a-4efe-b0eb-8a79c6093817 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b095b322f364d7c9f85d62a60d959ba, user_id=c3c91ce49691467a8aa79eb249c90277, role_id=3ca2dcbbd29d4a31a3ab78f2299bc239)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:33.390807 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fba9bd-506a-4efe-b0eb-8a79c6093817 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:33.406433 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1022/2063] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:45:33 2022] PUT /identity/v3/projects/4b095b322f364d7c9f85d62a60d959ba/users/c3c91ce49691467a8aa79eb249c90277/roles/3ca2dcbbd29d4a31a3ab78f2299bc239 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:45:33.410422 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0726201-c78a-4083-9c77-7f6371b4035f admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:33.423287 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0726201-c78a-4083-9c77-7f6371b4035f admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:33.431566 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fba9bd-506a-4efe-b0eb-8a79c6093817 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:33.433534 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-98fba9bd-506a-4efe-b0eb-8a79c6093817 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:45:33.485711 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:45:33.504260 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7a4d0b-c1e3-44d0-89ec-8e1be0ac9280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:33.505046 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7a4d0b-c1e3-44d0-89ec-8e1be0ac9280 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:33.505284 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7a4d0b-c1e3-44d0-89ec-8e1be0ac9280 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:33.505563 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7a4d0b-c1e3-44d0-89ec-8e1be0ac9280 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:33.506309 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7a4d0b-c1e3-44d0-89ec-8e1be0ac9280 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:33.507980 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7a4d0b-c1e3-44d0-89ec-8e1be0ac9280 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:33.513064 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fba9bd-506a-4efe-b0eb-8a79c6093817 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:33.517827 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3429c0-d905-4847-a364-0c75e75d029b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:33.518690 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3429c0-d905-4847-a364-0c75e75d029b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:33.518978 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3429c0-d905-4847-a364-0c75e75d029b admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:33.519091 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1042/2064] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:45:33 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:45:33.519322 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3429c0-d905-4847-a364-0c75e75d029b admin admin] PATH_INFO: `/v3/projects/b29bb73037b8496baaecbc9da5b21bca` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:33.520627 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3429c0-d905-4847-a364-0c75e75d029b admin admin] RBAC: Authorizing `identity:delete_project(project_id=b29bb73037b8496baaecbc9da5b21bca)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:33.522655 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3429c0-d905-4847-a364-0c75e75d029b admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:33.523112 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7a4d0b-c1e3-44d0-89ec-8e1be0ac9280 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:33.533540 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7a4d0b-c1e3-44d0-89ec-8e1be0ac9280 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:33.538425 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffdeae1-ab11-437a-8526-b0b2c7dade98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:33.539491 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffdeae1-ab11-437a-8526-b0b2c7dade98 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:33.539777 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffdeae1-ab11-437a-8526-b0b2c7dade98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:33.540057 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffdeae1-ab11-437a-8526-b0b2c7dade98 admin admin] PATH_INFO: `/v3/projects/4b095b322f364d7c9f85d62a60d959ba/users/c3c91ce49691467a8aa79eb249c90277/roles/3ca2dcbbd29d4a31a3ab78f2299bc239` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:33.542081 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffdeae1-ab11-437a-8526-b0b2c7dade98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b095b322f364d7c9f85d62a60d959ba, user_id=c3c91ce49691467a8aa79eb249c90277, role_id=3ca2dcbbd29d4a31a3ab78f2299bc239)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:33.545328 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffdeae1-ab11-437a-8526-b0b2c7dade98 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:33.557351 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1043/2065] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:45:33 2022] PUT /identity/v3/projects/4b095b322f364d7c9f85d62a60d959ba/users/c3c91ce49691467a8aa79eb249c90277/roles/3ca2dcbbd29d4a31a3ab78f2299bc239 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:45:33.568006 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdda86a-c718-4ce4-abac-8fa882a98b50 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:33.568386 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdda86a-c718-4ce4-abac-8fa882a98b50 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:33.568733 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdda86a-c718-4ce4-abac-8fa882a98b50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:33.573002 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-cc3429c0-d905-4847-a364-0c75e75d029b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b29bb73037b8496baaecbc9da5b21bca', 'request_id': 'req-cc3429c0-d905-4847-a364-0c75e75d029b'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:33.586931 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-cc3429c0-d905-4847-a364-0c75e75d029b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b29bb73037b8496baaecbc9da5b21bca', 'request_id': 'req-cc3429c0-d905-4847-a364-0c75e75d029b'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:33.595430 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-cc3429c0-d905-4847-a364-0c75e75d029b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b29bb73037b8496baaecbc9da5b21bca', 'request_id': 'req-cc3429c0-d905-4847-a364-0c75e75d029b'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:33.601282 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-cfdda86a-c718-4ce4-abac-8fa882a98b50 None None] MFA Rules not processed for user `c3c91ce49691467a8aa79eb249c90277`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:45:33.602532 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-cfdda86a-c718-4ce4-abac-8fa882a98b50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:45:33.607868 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-cc3429c0-d905-4847-a364-0c75e75d029b admin admin] The token cache is being invalidate because project b29bb73037b8496baaecbc9da5b21bca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80357) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:45:33.608153 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-cc3429c0-d905-4847-a364-0c75e75d029b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:33.608881 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-cc3429c0-d905-4847-a364-0c75e75d029b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:33.610526 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1023/2066] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jun 16 17:45:33 2022] DELETE /identity/v3/projects/b29bb73037b8496baaecbc9da5b21bca => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:45:33.687426 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1044/2067] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:45:33 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:45:33.700176 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3429c0-d905-4847-a364-0c75e75d029b admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:33.701958 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-cc3429c0-d905-4847-a364-0c75e75d029b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:45:33.740743 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:45:33.773036 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3429c0-d905-4847-a364-0c75e75d029b admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:33.776940 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaed264a-b3be-4a6d-b606-2289e01a0c5c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24d5e2637b7043e08313909d5b06e329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:33.778717 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaed264a-b3be-4a6d-b606-2289e01a0c5c service cinder] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:33.778717 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaed264a-b3be-4a6d-b606-2289e01a0c5c service cinder] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:33.778926 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaed264a-b3be-4a6d-b606-2289e01a0c5c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:33.789336 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaed264a-b3be-4a6d-b606-2289e01a0c5c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:33.790812 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaed264a-b3be-4a6d-b606-2289e01a0c5c service cinder] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:33.801934 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1024/2068] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:45:33 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:45:33.874566 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdda86a-c718-4ce4-abac-8fa882a98b50 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:33.886542 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdda86a-c718-4ce4-abac-8fa882a98b50 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:33.891427 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b05b95b-9d52-48d0-ae4b-9fd481a68ea3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24d5e2637b7043e08313909d5b06e329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:33.892209 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b05b95b-9d52-48d0-ae4b-9fd481a68ea3 service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:33.892684 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b05b95b-9d52-48d0-ae4b-9fd481a68ea3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:33.892973 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b05b95b-9d52-48d0-ae4b-9fd481a68ea3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:33.894763 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-8b05b95b-9d52-48d0-ae4b-9fd481a68ea3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:45:33.954634 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b05b95b-9d52-48d0-ae4b-9fd481a68ea3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:33.956070 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b05b95b-9d52-48d0-ae4b-9fd481a68ea3 service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:33.964974 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1045/2069] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:45:33 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:45:43.796888 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=13 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:45:43.902686 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaed264a-b3be-4a6d-b606-2289e01a0c5c service cinder] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:43.904094 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-aaed264a-b3be-4a6d-b606-2289e01a0c5c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:45:43.960804 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=77,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:45:43.997020 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaed264a-b3be-4a6d-b606-2289e01a0c5c service cinder] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:44.002169 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d97b0c-54d8-4c22-b6e4-48137708b707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:44.003118 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d97b0c-54d8-4c22-b6e4-48137708b707 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:44.003404 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d97b0c-54d8-4c22-b6e4-48137708b707 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:44.003686 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d97b0c-54d8-4c22-b6e4-48137708b707 admin admin] PATH_INFO: `/v3/users/23bbe5f1b48b4d3cb142a43bf7038df7` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:44.004910 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d97b0c-54d8-4c22-b6e4-48137708b707 admin admin] RBAC: Authorizing `identity:delete_user(user_id=23bbe5f1b48b4d3cb142a43bf7038df7)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:44.006834 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d97b0c-54d8-4c22-b6e4-48137708b707 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:44.054770 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-38d97b0c-54d8-4c22-b6e4-48137708b707 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23bbe5f1b48b4d3cb142a43bf7038df7'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:44.055430 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-38d97b0c-54d8-4c22-b6e4-48137708b707 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23bbe5f1b48b4d3cb142a43bf7038df7'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:44.064793 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-38d97b0c-54d8-4c22-b6e4-48137708b707 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23bbe5f1b48b4d3cb142a43bf7038df7'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:44.065457 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-38d97b0c-54d8-4c22-b6e4-48137708b707 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23bbe5f1b48b4d3cb142a43bf7038df7'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:44.079430 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-38d97b0c-54d8-4c22-b6e4-48137708b707 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23bbe5f1b48b4d3cb142a43bf7038df7'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:44.092811 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1025/2070] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jun 16 17:45:43 2022] DELETE /identity/v3/users/23bbe5f1b48b4d3cb142a43bf7038df7 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:45:44.097171 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b058d8d-b920-4b57-acfc-9c334197a4e3 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:44.097463 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b058d8d-b920-4b57-acfc-9c334197a4e3 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:44.097743 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b058d8d-b920-4b57-acfc-9c334197a4e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:44.116606 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-0b058d8d-b920-4b57-acfc-9c334197a4e3 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:45:44.117802 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-0b058d8d-b920-4b57-acfc-9c334197a4e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:45:44.177855 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1046/2071] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:45:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:45:44.187377 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d97b0c-54d8-4c22-b6e4-48137708b707 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:44.188823 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-38d97b0c-54d8-4c22-b6e4-48137708b707 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:45:44.248203 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d97b0c-54d8-4c22-b6e4-48137708b707 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:44.251810 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc2ef6c-07ea-418c-ad88-39cfd2a26442 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:44.252479 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc2ef6c-07ea-418c-ad88-39cfd2a26442 service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:44.252761 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc2ef6c-07ea-418c-ad88-39cfd2a26442 service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:44.253032 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc2ef6c-07ea-418c-ad88-39cfd2a26442 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:44.262081 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc2ef6c-07ea-418c-ad88-39cfd2a26442 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:44.263495 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc2ef6c-07ea-418c-ad88-39cfd2a26442 service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:44.273409 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1026/2072] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:45:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:45:44.547072 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b058d8d-b920-4b57-acfc-9c334197a4e3 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:44.548546 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-0b058d8d-b920-4b57-acfc-9c334197a4e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:45:44.562011 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b058d8d-b920-4b57-acfc-9c334197a4e3 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:44.566620 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae48c63-00d9-4b04-b2ec-60ad38aa6af5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:44.567470 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae48c63-00d9-4b04-b2ec-60ad38aa6af5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:44.567748 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae48c63-00d9-4b04-b2ec-60ad38aa6af5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:44.568018 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae48c63-00d9-4b04-b2ec-60ad38aa6af5 admin admin] PATH_INFO: `/v3/projects/1f044705a816445da5ede323dff39282` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:44.570075 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae48c63-00d9-4b04-b2ec-60ad38aa6af5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f044705a816445da5ede323dff39282)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:44.571981 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae48c63-00d9-4b04-b2ec-60ad38aa6af5 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:44.619116 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-0ae48c63-00d9-4b04-b2ec-60ad38aa6af5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f044705a816445da5ede323dff39282', 'request_id': 'req-0ae48c63-00d9-4b04-b2ec-60ad38aa6af5'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:44.626182 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-0ae48c63-00d9-4b04-b2ec-60ad38aa6af5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f044705a816445da5ede323dff39282', 'request_id': 'req-0ae48c63-00d9-4b04-b2ec-60ad38aa6af5'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:44.632613 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-0ae48c63-00d9-4b04-b2ec-60ad38aa6af5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f044705a816445da5ede323dff39282', 'request_id': 'req-0ae48c63-00d9-4b04-b2ec-60ad38aa6af5'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:44.643086 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-0ae48c63-00d9-4b04-b2ec-60ad38aa6af5 admin admin] The token cache is being invalidate because project 1f044705a816445da5ede323dff39282 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:45:44.643375 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-0ae48c63-00d9-4b04-b2ec-60ad38aa6af5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:44.644011 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-0ae48c63-00d9-4b04-b2ec-60ad38aa6af5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:44.645609 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1047/2073] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jun 16 17:45:44 2022] DELETE /identity/v3/projects/1f044705a816445da5ede323dff39282 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:45:44.649498 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc2ef6c-07ea-418c-ad88-39cfd2a26442 service neutron] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:44.659789 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc2ef6c-07ea-418c-ad88-39cfd2a26442 service neutron] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:44.678109 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:45:44.719837 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03593d7b-1bc1-4b90-ba51-a60974e15440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:44.720732 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03593d7b-1bc1-4b90-ba51-a60974e15440 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:44.721020 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03593d7b-1bc1-4b90-ba51-a60974e15440 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:44.721297 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03593d7b-1bc1-4b90-ba51-a60974e15440 admin admin] PATH_INFO: `/v3/users/c3c91ce49691467a8aa79eb249c90277` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:44.722477 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03593d7b-1bc1-4b90-ba51-a60974e15440 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3c91ce49691467a8aa79eb249c90277)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:44.724430 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03593d7b-1bc1-4b90-ba51-a60974e15440 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:44.776105 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-03593d7b-1bc1-4b90-ba51-a60974e15440 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3c91ce49691467a8aa79eb249c90277'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:44.776778 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-03593d7b-1bc1-4b90-ba51-a60974e15440 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3c91ce49691467a8aa79eb249c90277'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:44.785225 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-03593d7b-1bc1-4b90-ba51-a60974e15440 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3c91ce49691467a8aa79eb249c90277'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:44.786103 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-03593d7b-1bc1-4b90-ba51-a60974e15440 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3c91ce49691467a8aa79eb249c90277'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:44.799590 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-03593d7b-1bc1-4b90-ba51-a60974e15440 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3c91ce49691467a8aa79eb249c90277'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:44.812096 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1027/2074] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jun 16 17:45:44 2022] DELETE /identity/v3/users/c3c91ce49691467a8aa79eb249c90277 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:45:45.085748 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae48c63-00d9-4b04-b2ec-60ad38aa6af5 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:45.087409 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-0ae48c63-00d9-4b04-b2ec-60ad38aa6af5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:45:45.101621 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae48c63-00d9-4b04-b2ec-60ad38aa6af5 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:45.166887 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31638c6-d092-44ee-86dc-f0f02c640799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:45.167744 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31638c6-d092-44ee-86dc-f0f02c640799 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:45.168027 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31638c6-d092-44ee-86dc-f0f02c640799 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:45.168304 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31638c6-d092-44ee-86dc-f0f02c640799 admin admin] PATH_INFO: `/v3/projects/4b095b322f364d7c9f85d62a60d959ba` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:45.169552 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31638c6-d092-44ee-86dc-f0f02c640799 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b095b322f364d7c9f85d62a60d959ba)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:45.171998 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31638c6-d092-44ee-86dc-f0f02c640799 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:45.202124 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:45:45.220765 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-b31638c6-d092-44ee-86dc-f0f02c640799 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b095b322f364d7c9f85d62a60d959ba', 'request_id': 'req-b31638c6-d092-44ee-86dc-f0f02c640799'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:45.227834 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-b31638c6-d092-44ee-86dc-f0f02c640799 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b095b322f364d7c9f85d62a60d959ba', 'request_id': 'req-b31638c6-d092-44ee-86dc-f0f02c640799'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:45.234570 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-b31638c6-d092-44ee-86dc-f0f02c640799 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b095b322f364d7c9f85d62a60d959ba', 'request_id': 'req-b31638c6-d092-44ee-86dc-f0f02c640799'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:45.245140 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-b31638c6-d092-44ee-86dc-f0f02c640799 admin admin] The token cache is being invalidate because project 4b095b322f364d7c9f85d62a60d959ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:45:45.245428 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-b31638c6-d092-44ee-86dc-f0f02c640799 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:45.246067 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-b31638c6-d092-44ee-86dc-f0f02c640799 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:45.248656 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1048/2075] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jun 16 17:45:45 2022] DELETE /identity/v3/projects/4b095b322f364d7c9f85d62a60d959ba => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:45:45.253991 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455bb920-badf-4c08-ae73-77822fd0cd84 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:45.254368 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455bb920-badf-4c08-ae73-77822fd0cd84 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:45.254715 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455bb920-badf-4c08-ae73-77822fd0cd84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:45.273548 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-455bb920-badf-4c08-ae73-77822fd0cd84 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:45:45.274759 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-455bb920-badf-4c08-ae73-77822fd0cd84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:45:45.336309 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1028/2076] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:45:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:45:45.345167 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6549e0d-0228-4a97-8291-0217b972dfa1 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:45.345463 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6549e0d-0228-4a97-8291-0217b972dfa1 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:45.345741 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6549e0d-0228-4a97-8291-0217b972dfa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:45.363973 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-a6549e0d-0228-4a97-8291-0217b972dfa1 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:45:45.365222 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-a6549e0d-0228-4a97-8291-0217b972dfa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:45:45.373746 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1049/2077] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:45:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:45:45.378125 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455bb920-badf-4c08-ae73-77822fd0cd84 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:45.388150 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455bb920-badf-4c08-ae73-77822fd0cd84 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:45.395182 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b199b28-5f9a-4474-9e85-60359fe966f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:45.396227 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b199b28-5f9a-4474-9e85-60359fe966f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:45.396577 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b199b28-5f9a-4474-9e85-60359fe966f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:45.396901 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b199b28-5f9a-4474-9e85-60359fe966f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:45.397632 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b199b28-5f9a-4474-9e85-60359fe966f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:45:45.398059 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b199b28-5f9a-4474-9e85-60359fe966f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:45.399750 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b199b28-5f9a-4474-9e85-60359fe966f3 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:45.411499 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1029/2078] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jun 16 17:45:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:45:45.415744 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6549e0d-0228-4a97-8291-0217b972dfa1 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:45.425830 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6549e0d-0228-4a97-8291-0217b972dfa1 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:45.430586 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2fd83c-90fc-4783-8bf5-ceb541d7bbb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:45.431285 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2fd83c-90fc-4783-8bf5-ceb541d7bbb0 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:45.431591 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2fd83c-90fc-4783-8bf5-ceb541d7bbb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:45.431861 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2fd83c-90fc-4783-8bf5-ceb541d7bbb0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:45.432647 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2fd83c-90fc-4783-8bf5-ceb541d7bbb0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:45.434549 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2fd83c-90fc-4783-8bf5-ceb541d7bbb0 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:45.461635 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1050/2079] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jun 16 17:45:45 2022] POST /identity/v3/projects => generated 369 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:45:45.465860 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b199b28-5f9a-4474-9e85-60359fe966f3 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:45.476329 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b199b28-5f9a-4474-9e85-60359fe966f3 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:45.502468 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:45:45.535241 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8276195f-8984-4893-9682-432654f1dbc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:45.535910 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8276195f-8984-4893-9682-432654f1dbc8 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:45.536205 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8276195f-8984-4893-9682-432654f1dbc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:45.536529 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8276195f-8984-4893-9682-432654f1dbc8 admin admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:45.537223 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8276195f-8984-4893-9682-432654f1dbc8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:45.539137 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8276195f-8984-4893-9682-432654f1dbc8 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:45.560504 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1030/2080] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jun 16 17:45:45 2022] POST /identity/v3/users => generated 285 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:45:45.564960 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2fd83c-90fc-4783-8bf5-ceb541d7bbb0 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:45.575302 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2fd83c-90fc-4783-8bf5-ceb541d7bbb0 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:45.580084 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e10c37-271a-4ac4-9a2c-df27af2ae89d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:45.580780 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e10c37-271a-4ac4-9a2c-df27af2ae89d admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:45.581065 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e10c37-271a-4ac4-9a2c-df27af2ae89d admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:45.581343 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e10c37-271a-4ac4-9a2c-df27af2ae89d admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:45.582105 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e10c37-271a-4ac4-9a2c-df27af2ae89d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:45.583704 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e10c37-271a-4ac4-9a2c-df27af2ae89d admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:45.594676 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1051/2081] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:45:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:45:45.598389 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8276195f-8984-4893-9682-432654f1dbc8 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:45.609029 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8276195f-8984-4893-9682-432654f1dbc8 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:45.613602 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca65faf-5c4e-471a-97ff-2546fb966bca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:45.614254 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca65faf-5c4e-471a-97ff-2546fb966bca admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:45.614538 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca65faf-5c4e-471a-97ff-2546fb966bca admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:45.614815 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca65faf-5c4e-471a-97ff-2546fb966bca admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:45.615538 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca65faf-5c4e-471a-97ff-2546fb966bca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:45.617169 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca65faf-5c4e-471a-97ff-2546fb966bca admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:45.627295 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1031/2082] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:45:45 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:45:45.631308 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e10c37-271a-4ac4-9a2c-df27af2ae89d admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:45.641627 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e10c37-271a-4ac4-9a2c-df27af2ae89d admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:45.646208 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6d65cd-91fb-4781-a9b9-79aa995cdce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:45.647064 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6d65cd-91fb-4781-a9b9-79aa995cdce2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:45.647341 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6d65cd-91fb-4781-a9b9-79aa995cdce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:45.647612 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6d65cd-91fb-4781-a9b9-79aa995cdce2 admin admin] PATH_INFO: `/v3/projects/f52bb5856ae644b0a32245ad0323590d/users/2126abe80fc7459d8d264aee7f244697/roles/0be2075946f54c8485f1c275f746478a` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:45.663256 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6d65cd-91fb-4781-a9b9-79aa995cdce2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f52bb5856ae644b0a32245ad0323590d, user_id=2126abe80fc7459d8d264aee7f244697, role_id=0be2075946f54c8485f1c275f746478a)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:45.665947 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6d65cd-91fb-4781-a9b9-79aa995cdce2 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:45.679881 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1052/2083] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:45:45 2022] PUT /identity/v3/projects/f52bb5856ae644b0a32245ad0323590d/users/2126abe80fc7459d8d264aee7f244697/roles/0be2075946f54c8485f1c275f746478a => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:45:45.691545 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06889665-2040-4b1e-8cac-fd166e2f6264 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:45.691839 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06889665-2040-4b1e-8cac-fd166e2f6264 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:45.692130 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06889665-2040-4b1e-8cac-fd166e2f6264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:45.719045 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-06889665-2040-4b1e-8cac-fd166e2f6264 None None] MFA Rules not processed for user `2126abe80fc7459d8d264aee7f244697`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:45:45.720367 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-06889665-2040-4b1e-8cac-fd166e2f6264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:45:45.783261 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1032/2084] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:45:45 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:45:45.787701 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6d65cd-91fb-4781-a9b9-79aa995cdce2 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:45.848274 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6d65cd-91fb-4781-a9b9-79aa995cdce2 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:45.852908 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1098d565-2a57-48f3-90e0-c898ed427348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:45.853559 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1098d565-2a57-48f3-90e0-c898ed427348 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:45.853845 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1098d565-2a57-48f3-90e0-c898ed427348 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:45.854130 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1098d565-2a57-48f3-90e0-c898ed427348 admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:45.854826 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1098d565-2a57-48f3-90e0-c898ed427348 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:45.856760 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1098d565-2a57-48f3-90e0-c898ed427348 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:45.883619 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1053/2085] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jun 16 17:45:45 2022] POST /identity/v3/projects => generated 367 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:45:45.887421 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06889665-2040-4b1e-8cac-fd166e2f6264 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:45.898167 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06889665-2040-4b1e-8cac-fd166e2f6264 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:45.954627 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7eff46-b2ce-4478-9c6a-d13d519e4926 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:45.955297 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7eff46-b2ce-4478-9c6a-d13d519e4926 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:45.955579 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7eff46-b2ce-4478-9c6a-d13d519e4926 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:45.955850 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7eff46-b2ce-4478-9c6a-d13d519e4926 admin admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:45.956558 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7eff46-b2ce-4478-9c6a-d13d519e4926 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:45.958467 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7eff46-b2ce-4478-9c6a-d13d519e4926 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:45.978824 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1033/2086] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jun 16 17:45:45 2022] POST /identity/v3/users => generated 290 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:45:45.983611 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1098d565-2a57-48f3-90e0-c898ed427348 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:45.995375 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1098d565-2a57-48f3-90e0-c898ed427348 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:46.001163 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36191ce0-cd53-4f27-946d-40bf677f02db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:46.001834 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36191ce0-cd53-4f27-946d-40bf677f02db admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:46.002551 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36191ce0-cd53-4f27-946d-40bf677f02db admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:46.002551 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36191ce0-cd53-4f27-946d-40bf677f02db admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:46.003121 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36191ce0-cd53-4f27-946d-40bf677f02db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:46.004805 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36191ce0-cd53-4f27-946d-40bf677f02db admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:46.016836 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1054/2087] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:45:45 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 17:45:46.021064 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7eff46-b2ce-4478-9c6a-d13d519e4926 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:46.031301 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7eff46-b2ce-4478-9c6a-d13d519e4926 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:46.035855 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16879db3-680f-4641-a4fe-0d658cf0f2e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:46.036763 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16879db3-680f-4641-a4fe-0d658cf0f2e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:46.037061 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16879db3-680f-4641-a4fe-0d658cf0f2e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:46.037308 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16879db3-680f-4641-a4fe-0d658cf0f2e6 admin admin] PATH_INFO: `/v3/projects/1ca9da221d044ab0a316dfe143a597b2/users/0dba74e14d914609a4cdac9caedec115/roles/3ca2dcbbd29d4a31a3ab78f2299bc239` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:46.052527 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16879db3-680f-4641-a4fe-0d658cf0f2e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ca9da221d044ab0a316dfe143a597b2, user_id=0dba74e14d914609a4cdac9caedec115, role_id=3ca2dcbbd29d4a31a3ab78f2299bc239)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:46.056782 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16879db3-680f-4641-a4fe-0d658cf0f2e6 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:46.075167 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1034/2088] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:45:46 2022] PUT /identity/v3/projects/1ca9da221d044ab0a316dfe143a597b2/users/0dba74e14d914609a4cdac9caedec115/roles/3ca2dcbbd29d4a31a3ab78f2299bc239 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:45:46.079349 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36191ce0-cd53-4f27-946d-40bf677f02db admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:46.089724 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36191ce0-cd53-4f27-946d-40bf677f02db admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:46.110206 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:45:46.147987 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22f9d13-9227-413b-9ed9-85bda0e3033b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:46.148672 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22f9d13-9227-413b-9ed9-85bda0e3033b admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:46.148959 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22f9d13-9227-413b-9ed9-85bda0e3033b admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:46.149234 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22f9d13-9227-413b-9ed9-85bda0e3033b admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:46.149949 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22f9d13-9227-413b-9ed9-85bda0e3033b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:46.151536 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22f9d13-9227-413b-9ed9-85bda0e3033b admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:46.164359 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1055/2089] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:45:46 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:45:46.166044 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16879db3-680f-4641-a4fe-0d658cf0f2e6 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:46.171581 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:45:46.177103 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16879db3-680f-4641-a4fe-0d658cf0f2e6 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:46.183822 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb3e3cf-1f52-4028-ab74-0a7dc690f4ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:46.184738 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb3e3cf-1f52-4028-ab74-0a7dc690f4ca admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:46.185023 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb3e3cf-1f52-4028-ab74-0a7dc690f4ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:46.185301 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb3e3cf-1f52-4028-ab74-0a7dc690f4ca admin admin] PATH_INFO: `/v3/projects/1ca9da221d044ab0a316dfe143a597b2/users/0dba74e14d914609a4cdac9caedec115/roles/3ca2dcbbd29d4a31a3ab78f2299bc239` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:46.187791 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb3e3cf-1f52-4028-ab74-0a7dc690f4ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ca9da221d044ab0a316dfe143a597b2, user_id=0dba74e14d914609a4cdac9caedec115, role_id=3ca2dcbbd29d4a31a3ab78f2299bc239)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:46.190536 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb3e3cf-1f52-4028-ab74-0a7dc690f4ca admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:46.202086 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1035/2090] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:45:46 2022] PUT /identity/v3/projects/1ca9da221d044ab0a316dfe143a597b2/users/0dba74e14d914609a4cdac9caedec115/roles/3ca2dcbbd29d4a31a3ab78f2299bc239 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:45:46.212008 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06615526-ac1e-4e0e-9ce0-0fef5243bd26 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:46.212300 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06615526-ac1e-4e0e-9ce0-0fef5243bd26 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:46.212617 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06615526-ac1e-4e0e-9ce0-0fef5243bd26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:46.238508 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-06615526-ac1e-4e0e-9ce0-0fef5243bd26 None None] MFA Rules not processed for user `0dba74e14d914609a4cdac9caedec115`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:45:46.239709 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-06615526-ac1e-4e0e-9ce0-0fef5243bd26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:45:46.311231 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1056/2091] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:45:46 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:45:46.322889 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb3e3cf-1f52-4028-ab74-0a7dc690f4ca admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:46.324603 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-3bb3e3cf-1f52-4028-ab74-0a7dc690f4ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:45:46.385332 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb3e3cf-1f52-4028-ab74-0a7dc690f4ca admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:46.389844 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4181aa6a-9fe1-4133-93c1-37f468256c33 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24d5e2637b7043e08313909d5b06e329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:46.390479 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4181aa6a-9fe1-4133-93c1-37f468256c33 service cinder] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:46.390768 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4181aa6a-9fe1-4133-93c1-37f468256c33 service cinder] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:46.391041 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4181aa6a-9fe1-4133-93c1-37f468256c33 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:46.402316 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4181aa6a-9fe1-4133-93c1-37f468256c33 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:46.403733 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4181aa6a-9fe1-4133-93c1-37f468256c33 service cinder] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:46.414511 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1036/2092] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:45:46 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:45:47.780726 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06615526-ac1e-4e0e-9ce0-0fef5243bd26 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:47.840855 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06615526-ac1e-4e0e-9ce0-0fef5243bd26 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:47.845953 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566b86a4-575a-4bfe-be84-21e7d150fab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:47.846810 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566b86a4-575a-4bfe-be84-21e7d150fab8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:47.847090 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566b86a4-575a-4bfe-be84-21e7d150fab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:47.847381 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566b86a4-575a-4bfe-be84-21e7d150fab8 admin admin] PATH_INFO: `/v3/users/2126abe80fc7459d8d264aee7f244697` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:47.848582 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566b86a4-575a-4bfe-be84-21e7d150fab8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2126abe80fc7459d8d264aee7f244697)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:47.850484 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566b86a4-575a-4bfe-be84-21e7d150fab8 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:47.902761 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-566b86a4-575a-4bfe-be84-21e7d150fab8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2126abe80fc7459d8d264aee7f244697'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:47.903900 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:45:47.916797 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-566b86a4-575a-4bfe-be84-21e7d150fab8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2126abe80fc7459d8d264aee7f244697'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:47.917466 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-566b86a4-575a-4bfe-be84-21e7d150fab8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2126abe80fc7459d8d264aee7f244697'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:47.924594 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-566b86a4-575a-4bfe-be84-21e7d150fab8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2126abe80fc7459d8d264aee7f244697'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:47.931660 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-566b86a4-575a-4bfe-be84-21e7d150fab8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2126abe80fc7459d8d264aee7f244697'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:47.938108 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1057/2093] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jun 16 17:45:47 2022] DELETE /identity/v3/users/2126abe80fc7459d8d264aee7f244697 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:45:47.942744 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5adbc42-ec83-4d59-aa5f-516b4bae8262 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:47.943230 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5adbc42-ec83-4d59-aa5f-516b4bae8262 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:47.943335 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5adbc42-ec83-4d59-aa5f-516b4bae8262 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:47.961562 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-d5adbc42-ec83-4d59-aa5f-516b4bae8262 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:45:47.962767 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-d5adbc42-ec83-4d59-aa5f-516b4bae8262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:45:48.024569 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1037/2094] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:45:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:45:48.034508 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566b86a4-575a-4bfe-be84-21e7d150fab8 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:48.035945 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-566b86a4-575a-4bfe-be84-21e7d150fab8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:45:48.093912 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566b86a4-575a-4bfe-be84-21e7d150fab8 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:48.097592 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226f0f3a-acd6-4a97-a922-f160efe64080 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:48.098235 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226f0f3a-acd6-4a97-a922-f160efe64080 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:48.098520 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226f0f3a-acd6-4a97-a922-f160efe64080 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:48.098800 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226f0f3a-acd6-4a97-a922-f160efe64080 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:48.108761 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226f0f3a-acd6-4a97-a922-f160efe64080 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:48.110185 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226f0f3a-acd6-4a97-a922-f160efe64080 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:48.119653 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1058/2095] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:45:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:45:48.396334 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5adbc42-ec83-4d59-aa5f-516b4bae8262 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:48.397897 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-d5adbc42-ec83-4d59-aa5f-516b4bae8262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:45:48.428503 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5adbc42-ec83-4d59-aa5f-516b4bae8262 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:48.447244 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bc312c-b35a-40b6-a4c5-59f72c42bf8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:48.448116 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bc312c-b35a-40b6-a4c5-59f72c42bf8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:48.448416 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bc312c-b35a-40b6-a4c5-59f72c42bf8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:48.448713 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bc312c-b35a-40b6-a4c5-59f72c42bf8b admin admin] PATH_INFO: `/v3/projects/f52bb5856ae644b0a32245ad0323590d` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:48.449900 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bc312c-b35a-40b6-a4c5-59f72c42bf8b admin admin] RBAC: Authorizing `identity:delete_project(project_id=f52bb5856ae644b0a32245ad0323590d)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:48.452006 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bc312c-b35a-40b6-a4c5-59f72c42bf8b admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:48.499973 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-88bc312c-b35a-40b6-a4c5-59f72c42bf8b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f52bb5856ae644b0a32245ad0323590d', 'request_id': 'req-88bc312c-b35a-40b6-a4c5-59f72c42bf8b'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:48.506939 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-88bc312c-b35a-40b6-a4c5-59f72c42bf8b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f52bb5856ae644b0a32245ad0323590d', 'request_id': 'req-88bc312c-b35a-40b6-a4c5-59f72c42bf8b'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:48.512891 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-88bc312c-b35a-40b6-a4c5-59f72c42bf8b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f52bb5856ae644b0a32245ad0323590d', 'request_id': 'req-88bc312c-b35a-40b6-a4c5-59f72c42bf8b'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:48.524358 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-88bc312c-b35a-40b6-a4c5-59f72c42bf8b admin admin] The token cache is being invalidate because project f52bb5856ae644b0a32245ad0323590d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80357) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:45:48.524679 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-88bc312c-b35a-40b6-a4c5-59f72c42bf8b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:48.525311 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-88bc312c-b35a-40b6-a4c5-59f72c42bf8b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:48.526847 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1038/2096] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jun 16 17:45:48 2022] DELETE /identity/v3/projects/f52bb5856ae644b0a32245ad0323590d => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:45:48.531484 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226f0f3a-acd6-4a97-a922-f160efe64080 service neutron] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:48.541047 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226f0f3a-acd6-4a97-a922-f160efe64080 service neutron] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:48.601990 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672021d0-cbbc-4b54-8d71-619ebf05e390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:48.602908 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672021d0-cbbc-4b54-8d71-619ebf05e390 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:48.603193 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672021d0-cbbc-4b54-8d71-619ebf05e390 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:48.603473 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672021d0-cbbc-4b54-8d71-619ebf05e390 admin admin] PATH_INFO: `/v3/users/0dba74e14d914609a4cdac9caedec115` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:48.604773 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672021d0-cbbc-4b54-8d71-619ebf05e390 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0dba74e14d914609a4cdac9caedec115)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:48.606701 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672021d0-cbbc-4b54-8d71-619ebf05e390 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:48.656458 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-672021d0-cbbc-4b54-8d71-619ebf05e390 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0dba74e14d914609a4cdac9caedec115'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:48.668585 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-672021d0-cbbc-4b54-8d71-619ebf05e390 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0dba74e14d914609a4cdac9caedec115'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:48.669366 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-672021d0-cbbc-4b54-8d71-619ebf05e390 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0dba74e14d914609a4cdac9caedec115'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:48.674490 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:45:48.677619 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-672021d0-cbbc-4b54-8d71-619ebf05e390 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0dba74e14d914609a4cdac9caedec115'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:48.686320 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-672021d0-cbbc-4b54-8d71-619ebf05e390 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0dba74e14d914609a4cdac9caedec115'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:48.692366 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1059/2097] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jun 16 17:45:48 2022] DELETE /identity/v3/users/0dba74e14d914609a4cdac9caedec115 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:45:48.962912 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bc312c-b35a-40b6-a4c5-59f72c42bf8b admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:48.964503 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-88bc312c-b35a-40b6-a4c5-59f72c42bf8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:45:48.980665 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bc312c-b35a-40b6-a4c5-59f72c42bf8b admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:48.994043 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:45:49.041451 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd713d5-b545-40f1-8896-a31a5947468d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:49.042236 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd713d5-b545-40f1-8896-a31a5947468d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:49.042519 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd713d5-b545-40f1-8896-a31a5947468d admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:49.042792 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd713d5-b545-40f1-8896-a31a5947468d admin admin] PATH_INFO: `/v3/projects/1ca9da221d044ab0a316dfe143a597b2` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:49.043962 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd713d5-b545-40f1-8896-a31a5947468d admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ca9da221d044ab0a316dfe143a597b2)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:49.046025 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd713d5-b545-40f1-8896-a31a5947468d admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:49.091437 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-cdd713d5-b545-40f1-8896-a31a5947468d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ca9da221d044ab0a316dfe143a597b2', 'request_id': 'req-cdd713d5-b545-40f1-8896-a31a5947468d'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:49.104892 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-cdd713d5-b545-40f1-8896-a31a5947468d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ca9da221d044ab0a316dfe143a597b2', 'request_id': 'req-cdd713d5-b545-40f1-8896-a31a5947468d'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:49.110788 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-cdd713d5-b545-40f1-8896-a31a5947468d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ca9da221d044ab0a316dfe143a597b2', 'request_id': 'req-cdd713d5-b545-40f1-8896-a31a5947468d'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:49.122455 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-cdd713d5-b545-40f1-8896-a31a5947468d admin admin] The token cache is being invalidate because project 1ca9da221d044ab0a316dfe143a597b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80357) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:45:49.123298 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-cdd713d5-b545-40f1-8896-a31a5947468d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:49.124105 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-cdd713d5-b545-40f1-8896-a31a5947468d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:49.125709 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1039/2098] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jun 16 17:45:48 2022] DELETE /identity/v3/projects/1ca9da221d044ab0a316dfe143a597b2 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:45:49.135510 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753866c0-e627-4782-aa3d-4d7e4f8c2ec9 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:49.136016 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753866c0-e627-4782-aa3d-4d7e4f8c2ec9 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:49.136114 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753866c0-e627-4782-aa3d-4d7e4f8c2ec9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:49.156745 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-753866c0-e627-4782-aa3d-4d7e4f8c2ec9 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:45:49.158045 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-753866c0-e627-4782-aa3d-4d7e4f8c2ec9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:45:49.224015 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1060/2099] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:45:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:45:49.231755 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49a9a74-8f33-4321-ac95-03db28b587cb None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:49.232042 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49a9a74-8f33-4321-ac95-03db28b587cb None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:49.232360 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49a9a74-8f33-4321-ac95-03db28b587cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:49.250915 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-e49a9a74-8f33-4321-ac95-03db28b587cb None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:45:49.252107 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-e49a9a74-8f33-4321-ac95-03db28b587cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:45:49.260350 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1040/2100] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:45:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:45:49.265373 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753866c0-e627-4782-aa3d-4d7e4f8c2ec9 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:49.275182 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753866c0-e627-4782-aa3d-4d7e4f8c2ec9 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:49.281407 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeb8fdf-a373-4cba-8df6-b3822fb841c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:49.282082 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeb8fdf-a373-4cba-8df6-b3822fb841c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:49.282417 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeb8fdf-a373-4cba-8df6-b3822fb841c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:49.282691 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeb8fdf-a373-4cba-8df6-b3822fb841c6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:49.283416 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeeb8fdf-a373-4cba-8df6-b3822fb841c6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:45:49.283725 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeeb8fdf-a373-4cba-8df6-b3822fb841c6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:49.285377 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeeb8fdf-a373-4cba-8df6-b3822fb841c6 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:49.297801 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1061/2101] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jun 16 17:45:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:45:49.303323 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49a9a74-8f33-4321-ac95-03db28b587cb None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:49.313943 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49a9a74-8f33-4321-ac95-03db28b587cb None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:49.319129 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ca8d16-5660-4aec-815b-32fc2bbd4bac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:49.324663 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ca8d16-5660-4aec-815b-32fc2bbd4bac admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:49.324965 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ca8d16-5660-4aec-815b-32fc2bbd4bac admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:49.325247 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ca8d16-5660-4aec-815b-32fc2bbd4bac admin admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:49.326739 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ca8d16-5660-4aec-815b-32fc2bbd4bac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:49.329116 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ca8d16-5660-4aec-815b-32fc2bbd4bac admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:49.363166 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1041/2102] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jun 16 17:45:49 2022] POST /identity/v3/projects => generated 387 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:45:49.365507 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeb8fdf-a373-4cba-8df6-b3822fb841c6 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:49.375934 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeb8fdf-a373-4cba-8df6-b3822fb841c6 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:49.436681 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb12336-79a0-4e1a-842c-d33276079586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:49.437393 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb12336-79a0-4e1a-842c-d33276079586 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:49.437704 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb12336-79a0-4e1a-842c-d33276079586 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:49.437959 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb12336-79a0-4e1a-842c-d33276079586 admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:49.438667 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb12336-79a0-4e1a-842c-d33276079586 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:49.440629 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb12336-79a0-4e1a-842c-d33276079586 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:49.469062 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1062/2103] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jun 16 17:45:49 2022] POST /identity/v3/users => generated 294 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:45:49.472942 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ca8d16-5660-4aec-815b-32fc2bbd4bac admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:49.487260 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ca8d16-5660-4aec-815b-32fc2bbd4bac admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:49.494660 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51b6c81-dc06-46b2-a822-ceb574d6fae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:49.495551 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51b6c81-dc06-46b2-a822-ceb574d6fae5 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:49.495897 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51b6c81-dc06-46b2-a822-ceb574d6fae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:49.497308 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51b6c81-dc06-46b2-a822-ceb574d6fae5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:49.498275 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51b6c81-dc06-46b2-a822-ceb574d6fae5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:49.499615 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51b6c81-dc06-46b2-a822-ceb574d6fae5 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:49.519667 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1042/2104] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:45:49 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:45:49.524249 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb12336-79a0-4e1a-842c-d33276079586 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:49.535787 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb12336-79a0-4e1a-842c-d33276079586 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:49.541391 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e176bf5b-e65b-4b79-8d1f-3e007a92392c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:49.542070 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e176bf5b-e65b-4b79-8d1f-3e007a92392c admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:49.542351 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e176bf5b-e65b-4b79-8d1f-3e007a92392c admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:49.542632 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e176bf5b-e65b-4b79-8d1f-3e007a92392c admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:49.543364 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e176bf5b-e65b-4b79-8d1f-3e007a92392c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:49.545369 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e176bf5b-e65b-4b79-8d1f-3e007a92392c admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:49.556589 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1063/2105] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:45:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:45:49.560941 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51b6c81-dc06-46b2-a822-ceb574d6fae5 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:49.574443 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51b6c81-dc06-46b2-a822-ceb574d6fae5 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:49.581435 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fed4ae6-6550-4b71-bb89-afd7df426291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:49.586793 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fed4ae6-6550-4b71-bb89-afd7df426291 admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:49.586793 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fed4ae6-6550-4b71-bb89-afd7df426291 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:49.586793 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fed4ae6-6550-4b71-bb89-afd7df426291 admin admin] PATH_INFO: `/v3/projects/9e08a72841c641c6ae81856a713ed967/users/bd919abbe5ce4e07951be9e26c48da56/roles/0be2075946f54c8485f1c275f746478a` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:49.602481 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fed4ae6-6550-4b71-bb89-afd7df426291 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e08a72841c641c6ae81856a713ed967, user_id=bd919abbe5ce4e07951be9e26c48da56, role_id=0be2075946f54c8485f1c275f746478a)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:49.605631 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fed4ae6-6550-4b71-bb89-afd7df426291 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:49.632445 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1043/2106] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:45:49 2022] PUT /identity/v3/projects/9e08a72841c641c6ae81856a713ed967/users/bd919abbe5ce4e07951be9e26c48da56/roles/0be2075946f54c8485f1c275f746478a => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:45:49.639127 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e13efe8-8442-4b0d-b2a0-cab7722dce61 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:49.640187 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e13efe8-8442-4b0d-b2a0-cab7722dce61 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:49.641583 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e13efe8-8442-4b0d-b2a0-cab7722dce61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:49.683405 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-5e13efe8-8442-4b0d-b2a0-cab7722dce61 None None] MFA Rules not processed for user `bd919abbe5ce4e07951be9e26c48da56`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:45:49.684883 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-5e13efe8-8442-4b0d-b2a0-cab7722dce61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:45:49.737949 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:45:49.766699 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1064/2107] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:45:49 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:45:49.771270 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fed4ae6-6550-4b71-bb89-afd7df426291 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:49.819510 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:45:49.839359 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fed4ae6-6550-4b71-bb89-afd7df426291 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:49.844834 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0cb90d-65b1-4eb9-a9b4-4d3c249ebbcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:49.845489 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0cb90d-65b1-4eb9-a9b4-4d3c249ebbcd admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:49.845775 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0cb90d-65b1-4eb9-a9b4-4d3c249ebbcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:49.846200 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0cb90d-65b1-4eb9-a9b4-4d3c249ebbcd admin admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:49.846723 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0cb90d-65b1-4eb9-a9b4-4d3c249ebbcd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:49.848968 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0cb90d-65b1-4eb9-a9b4-4d3c249ebbcd admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:49.875928 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1044/2108] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jun 16 17:45:49 2022] POST /identity/v3/projects => generated 385 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:45:49.879845 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e13efe8-8442-4b0d-b2a0-cab7722dce61 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:49.892089 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e13efe8-8442-4b0d-b2a0-cab7722dce61 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:49.952798 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9807056-40db-4cd7-84f2-ddd60222555b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:49.953467 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9807056-40db-4cd7-84f2-ddd60222555b admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:49.953746 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9807056-40db-4cd7-84f2-ddd60222555b admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:49.954021 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9807056-40db-4cd7-84f2-ddd60222555b admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:49.954702 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9807056-40db-4cd7-84f2-ddd60222555b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:49.956645 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9807056-40db-4cd7-84f2-ddd60222555b admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:49.978340 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1065/2109] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jun 16 17:45:49 2022] POST /identity/v3/users => generated 293 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:45:49.982604 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0cb90d-65b1-4eb9-a9b4-4d3c249ebbcd admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:49.994373 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0cb90d-65b1-4eb9-a9b4-4d3c249ebbcd admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:50.000074 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e6d606-c2bc-436d-9f2a-d77249787c77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:50.000785 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e6d606-c2bc-436d-9f2a-d77249787c77 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:50.001133 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e6d606-c2bc-436d-9f2a-d77249787c77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:50.001537 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e6d606-c2bc-436d-9f2a-d77249787c77 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:50.002474 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e6d606-c2bc-436d-9f2a-d77249787c77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:50.004069 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e6d606-c2bc-436d-9f2a-d77249787c77 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:50.018430 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1045/2110] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:45:49 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:45:50.022639 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9807056-40db-4cd7-84f2-ddd60222555b admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:50.033000 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9807056-40db-4cd7-84f2-ddd60222555b admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:50.038430 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f58544f-d7a1-44c8-b006-7dd77d15a64e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:50.039097 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f58544f-d7a1-44c8-b006-7dd77d15a64e admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:50.039385 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f58544f-d7a1-44c8-b006-7dd77d15a64e admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:50.039664 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f58544f-d7a1-44c8-b006-7dd77d15a64e admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:50.040521 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f58544f-d7a1-44c8-b006-7dd77d15a64e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:50.042123 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f58544f-d7a1-44c8-b006-7dd77d15a64e admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:50.052738 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1066/2111] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:45:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:45:50.057547 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e6d606-c2bc-436d-9f2a-d77249787c77 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:50.070680 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e6d606-c2bc-436d-9f2a-d77249787c77 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:50.077057 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f3a1aa-a6dc-484a-b8c8-f00f521f936a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:50.078042 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f3a1aa-a6dc-484a-b8c8-f00f521f936a admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:50.078332 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f3a1aa-a6dc-484a-b8c8-f00f521f936a admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:50.078609 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f3a1aa-a6dc-484a-b8c8-f00f521f936a admin admin] PATH_INFO: `/v3/projects/9d4a920f844d410eb6e42ae302d134c2/users/8f47b5649e484ba1966df56ae838620f/roles/0be2075946f54c8485f1c275f746478a` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:50.095449 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f3a1aa-a6dc-484a-b8c8-f00f521f936a admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d4a920f844d410eb6e42ae302d134c2, user_id=8f47b5649e484ba1966df56ae838620f, role_id=0be2075946f54c8485f1c275f746478a)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:50.098389 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f3a1aa-a6dc-484a-b8c8-f00f521f936a admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:50.113596 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1046/2112] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:45:50 2022] PUT /identity/v3/projects/9d4a920f844d410eb6e42ae302d134c2/users/8f47b5649e484ba1966df56ae838620f/roles/0be2075946f54c8485f1c275f746478a => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:45:50.124784 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec86244c-ffea-4a19-9d12-2a5e4cb5f766 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:50.125422 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec86244c-ffea-4a19-9d12-2a5e4cb5f766 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:50.125913 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec86244c-ffea-4a19-9d12-2a5e4cb5f766 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:50.184806 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-ec86244c-ffea-4a19-9d12-2a5e4cb5f766 None None] MFA Rules not processed for user `8f47b5649e484ba1966df56ae838620f`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:45:50.186683 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-ec86244c-ffea-4a19-9d12-2a5e4cb5f766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:45:50.259274 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1067/2113] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:45:50 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:45:50.263855 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f3a1aa-a6dc-484a-b8c8-f00f521f936a admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:50.328321 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f3a1aa-a6dc-484a-b8c8-f00f521f936a admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:50.333175 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b99e4ce-ea07-42e1-a09e-69f240e3d2e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:50.333898 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b99e4ce-ea07-42e1-a09e-69f240e3d2e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:50.334182 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b99e4ce-ea07-42e1-a09e-69f240e3d2e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:50.334461 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b99e4ce-ea07-42e1-a09e-69f240e3d2e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:50.335178 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b99e4ce-ea07-42e1-a09e-69f240e3d2e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:50.337222 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b99e4ce-ea07-42e1-a09e-69f240e3d2e1 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:50.374459 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1047/2114] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jun 16 17:45:50 2022] POST /identity/v3/projects => generated 385 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:45:50.378779 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec86244c-ffea-4a19-9d12-2a5e4cb5f766 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:50.395903 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec86244c-ffea-4a19-9d12-2a5e4cb5f766 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:50.452962 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:45:50.478309 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1a7fe7-e9a5-4ac9-a656-5a502b8e9d04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:50.478309 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1a7fe7-e9a5-4ac9-a656-5a502b8e9d04 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:50.478309 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1a7fe7-e9a5-4ac9-a656-5a502b8e9d04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:50.478309 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1a7fe7-e9a5-4ac9-a656-5a502b8e9d04 admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:50.479059 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1a7fe7-e9a5-4ac9-a656-5a502b8e9d04 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:50.479059 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1a7fe7-e9a5-4ac9-a656-5a502b8e9d04 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:50.503383 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1068/2115] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jun 16 17:45:50 2022] POST /identity/v3/users => generated 299 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:45:50.507363 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b99e4ce-ea07-42e1-a09e-69f240e3d2e1 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:50.522058 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b99e4ce-ea07-42e1-a09e-69f240e3d2e1 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:50.530302 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e941cd67-9871-4f08-ba3c-4e387ee95fd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:50.531209 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e941cd67-9871-4f08-ba3c-4e387ee95fd9 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:50.531594 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e941cd67-9871-4f08-ba3c-4e387ee95fd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:50.531996 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e941cd67-9871-4f08-ba3c-4e387ee95fd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:50.532869 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e941cd67-9871-4f08-ba3c-4e387ee95fd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:50.535447 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e941cd67-9871-4f08-ba3c-4e387ee95fd9 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:50.552025 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1048/2116] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:45:50 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 17:45:50.555949 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1a7fe7-e9a5-4ac9-a656-5a502b8e9d04 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:50.567676 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1a7fe7-e9a5-4ac9-a656-5a502b8e9d04 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:50.574519 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebfc51d-38e8-491b-8595-b8a1461d1408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:50.576825 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebfc51d-38e8-491b-8595-b8a1461d1408 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:50.577257 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebfc51d-38e8-491b-8595-b8a1461d1408 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:50.577608 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebfc51d-38e8-491b-8595-b8a1461d1408 admin admin] PATH_INFO: `/v3/projects/b77a6911f549462d8e458da87438003c/users/32b1ece24aea460d8bb5c30e927ac5e5/roles/3ca2dcbbd29d4a31a3ab78f2299bc239` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:50.608985 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebfc51d-38e8-491b-8595-b8a1461d1408 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b77a6911f549462d8e458da87438003c, user_id=32b1ece24aea460d8bb5c30e927ac5e5, role_id=3ca2dcbbd29d4a31a3ab78f2299bc239)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:50.611722 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebfc51d-38e8-491b-8595-b8a1461d1408 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:50.633011 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1069/2117] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:45:50 2022] PUT /identity/v3/projects/b77a6911f549462d8e458da87438003c/users/32b1ece24aea460d8bb5c30e927ac5e5/roles/3ca2dcbbd29d4a31a3ab78f2299bc239 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:45:50.637430 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e941cd67-9871-4f08-ba3c-4e387ee95fd9 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:50.649210 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e941cd67-9871-4f08-ba3c-4e387ee95fd9 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:50.728321 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c338072d-703a-45ae-8477-566e65fd8837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:50.729078 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c338072d-703a-45ae-8477-566e65fd8837 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:50.729409 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c338072d-703a-45ae-8477-566e65fd8837 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:50.730264 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c338072d-703a-45ae-8477-566e65fd8837 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:50.730536 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c338072d-703a-45ae-8477-566e65fd8837 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:50.732658 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c338072d-703a-45ae-8477-566e65fd8837 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:50.759501 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1049/2118] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:45:50 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:45:50.765901 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebfc51d-38e8-491b-8595-b8a1461d1408 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:50.781250 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebfc51d-38e8-491b-8595-b8a1461d1408 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:50.787119 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e1d03c-6084-4604-8d0b-90a19d8e2d82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:50.788121 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e1d03c-6084-4604-8d0b-90a19d8e2d82 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:50.788574 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e1d03c-6084-4604-8d0b-90a19d8e2d82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:50.788896 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e1d03c-6084-4604-8d0b-90a19d8e2d82 admin admin] PATH_INFO: `/v3/projects/b77a6911f549462d8e458da87438003c/users/32b1ece24aea460d8bb5c30e927ac5e5/roles/3ca2dcbbd29d4a31a3ab78f2299bc239` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:50.790972 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e1d03c-6084-4604-8d0b-90a19d8e2d82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b77a6911f549462d8e458da87438003c, user_id=32b1ece24aea460d8bb5c30e927ac5e5, role_id=3ca2dcbbd29d4a31a3ab78f2299bc239)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:50.801251 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e1d03c-6084-4604-8d0b-90a19d8e2d82 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:50.816636 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1070/2119] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:45:50 2022] PUT /identity/v3/projects/b77a6911f549462d8e458da87438003c/users/32b1ece24aea460d8bb5c30e927ac5e5/roles/3ca2dcbbd29d4a31a3ab78f2299bc239 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:45:50.827716 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e0b10b-5ddf-4801-b3ad-c5780982efe2 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:50.828138 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e0b10b-5ddf-4801-b3ad-c5780982efe2 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:50.828556 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e0b10b-5ddf-4801-b3ad-c5780982efe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:50.837249 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:45:50.882281 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-15e0b10b-5ddf-4801-b3ad-c5780982efe2 None None] MFA Rules not processed for user `32b1ece24aea460d8bb5c30e927ac5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:45:50.883755 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-15e0b10b-5ddf-4801-b3ad-c5780982efe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:45:50.977129 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1050/2120] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:45:50 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:45:50.993026 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e1d03c-6084-4604-8d0b-90a19d8e2d82 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:50.994748 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-27e1d03c-6084-4604-8d0b-90a19d8e2d82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:45:51.058627 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e1d03c-6084-4604-8d0b-90a19d8e2d82 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:51.064365 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7eb625-2973-4665-b8c7-479440fa502f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24d5e2637b7043e08313909d5b06e329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:51.065327 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7eb625-2973-4665-b8c7-479440fa502f service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:51.065684 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7eb625-2973-4665-b8c7-479440fa502f service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:51.066051 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7eb625-2973-4665-b8c7-479440fa502f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:51.076385 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7eb625-2973-4665-b8c7-479440fa502f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:51.078046 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7eb625-2973-4665-b8c7-479440fa502f service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:51.088148 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1071/2121] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:45:50 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:45:51.611076 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e0b10b-5ddf-4801-b3ad-c5780982efe2 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:51.623417 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e0b10b-5ddf-4801-b3ad-c5780982efe2 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:51.627709 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c8b11f-3103-4747-b6d5-af41e8669661 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24d5e2637b7043e08313909d5b06e329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:51.628424 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c8b11f-3103-4747-b6d5-af41e8669661 service cinder] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:51.628763 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c8b11f-3103-4747-b6d5-af41e8669661 service cinder] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:51.629047 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c8b11f-3103-4747-b6d5-af41e8669661 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:51.701881 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c8b11f-3103-4747-b6d5-af41e8669661 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:51.703504 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c8b11f-3103-4747-b6d5-af41e8669661 service cinder] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:51.715193 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1051/2122] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:45:51 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:45:52.069778 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7eb625-2973-4665-b8c7-479440fa502f service cinder] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:52.071289 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-ce7eb625-2973-4665-b8c7-479440fa502f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:45:52.146205 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7eb625-2973-4665-b8c7-479440fa502f service cinder] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:52.152318 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0214932e-cdca-454b-a734-f7c1d141f64d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:52.153237 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0214932e-cdca-454b-a734-f7c1d141f64d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:52.153525 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0214932e-cdca-454b-a734-f7c1d141f64d admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:52.153800 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0214932e-cdca-454b-a734-f7c1d141f64d admin admin] PATH_INFO: `/v3/users/416a93006e65481c82791c17e04a0fae` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:52.156016 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0214932e-cdca-454b-a734-f7c1d141f64d admin admin] RBAC: Authorizing `identity:delete_user(user_id=416a93006e65481c82791c17e04a0fae)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:52.159376 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0214932e-cdca-454b-a734-f7c1d141f64d admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:52.214700 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-0214932e-cdca-454b-a734-f7c1d141f64d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '416a93006e65481c82791c17e04a0fae'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:52.216155 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:45:52.237491 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-0214932e-cdca-454b-a734-f7c1d141f64d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '416a93006e65481c82791c17e04a0fae'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:52.238588 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-0214932e-cdca-454b-a734-f7c1d141f64d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '416a93006e65481c82791c17e04a0fae'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:52.250204 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-0214932e-cdca-454b-a734-f7c1d141f64d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '416a93006e65481c82791c17e04a0fae'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:52.260502 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-0214932e-cdca-454b-a734-f7c1d141f64d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '416a93006e65481c82791c17e04a0fae'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:52.268453 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1072/2123] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jun 16 17:45:52 2022] DELETE /identity/v3/users/416a93006e65481c82791c17e04a0fae => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:45:52.273359 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f03941-1543-4790-a8c6-cf7b132fc7b8 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:52.273656 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f03941-1543-4790-a8c6-cf7b132fc7b8 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:52.273940 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f03941-1543-4790-a8c6-cf7b132fc7b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:52.296496 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-d9f03941-1543-4790-a8c6-cf7b132fc7b8 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:45:52.297973 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-d9f03941-1543-4790-a8c6-cf7b132fc7b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:45:52.370246 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1052/2124] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:45:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:45:52.382131 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0214932e-cdca-454b-a734-f7c1d141f64d admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:52.383897 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-0214932e-cdca-454b-a734-f7c1d141f64d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:45:52.454113 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0214932e-cdca-454b-a734-f7c1d141f64d admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:52.458256 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dc50b2-fdc2-4164-b2ca-440ec04ba1d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:52.458925 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dc50b2-fdc2-4164-b2ca-440ec04ba1d1 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:52.459216 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dc50b2-fdc2-4164-b2ca-440ec04ba1d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:52.459490 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dc50b2-fdc2-4164-b2ca-440ec04ba1d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:52.472466 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dc50b2-fdc2-4164-b2ca-440ec04ba1d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:52.474008 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dc50b2-fdc2-4164-b2ca-440ec04ba1d1 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:52.484600 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1073/2125] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:45:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:45:52.764068 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f03941-1543-4790-a8c6-cf7b132fc7b8 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:52.765614 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-d9f03941-1543-4790-a8c6-cf7b132fc7b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:45:52.778549 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f03941-1543-4790-a8c6-cf7b132fc7b8 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:52.784568 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e740b058-b079-48d9-a9fd-14987e6c5b6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:52.785610 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e740b058-b079-48d9-a9fd-14987e6c5b6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:52.785897 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e740b058-b079-48d9-a9fd-14987e6c5b6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:52.786239 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e740b058-b079-48d9-a9fd-14987e6c5b6d admin admin] PATH_INFO: `/v3/projects/774476afd17b44b1bae76c13a300577e` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:52.787388 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e740b058-b079-48d9-a9fd-14987e6c5b6d admin admin] RBAC: Authorizing `identity:delete_project(project_id=774476afd17b44b1bae76c13a300577e)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:52.789333 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e740b058-b079-48d9-a9fd-14987e6c5b6d admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:52.812863 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:45:52.838745 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-e740b058-b079-48d9-a9fd-14987e6c5b6d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '774476afd17b44b1bae76c13a300577e', 'request_id': 'req-e740b058-b079-48d9-a9fd-14987e6c5b6d'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:52.847433 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-e740b058-b079-48d9-a9fd-14987e6c5b6d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '774476afd17b44b1bae76c13a300577e', 'request_id': 'req-e740b058-b079-48d9-a9fd-14987e6c5b6d'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:52.853606 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-e740b058-b079-48d9-a9fd-14987e6c5b6d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '774476afd17b44b1bae76c13a300577e', 'request_id': 'req-e740b058-b079-48d9-a9fd-14987e6c5b6d'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:52.865072 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-e740b058-b079-48d9-a9fd-14987e6c5b6d admin admin] The token cache is being invalidate because project 774476afd17b44b1bae76c13a300577e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80357) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:45:52.865429 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-e740b058-b079-48d9-a9fd-14987e6c5b6d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:52.866005 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-e740b058-b079-48d9-a9fd-14987e6c5b6d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:52.867930 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1053/2126] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jun 16 17:45:52 2022] DELETE /identity/v3/projects/774476afd17b44b1bae76c13a300577e => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:45:52.873372 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f180b13d-d7ca-4910-9c20-bbf4a6c62537 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:52.873648 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f180b13d-d7ca-4910-9c20-bbf4a6c62537 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:52.873936 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f180b13d-d7ca-4910-9c20-bbf4a6c62537 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:52.894563 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-f180b13d-d7ca-4910-9c20-bbf4a6c62537 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:45:52.895773 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-f180b13d-d7ca-4910-9c20-bbf4a6c62537 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:45:52.957096 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1074/2127] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:45:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:45:52.965026 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d547eba8-9973-475f-aac6-6a4c43961eec None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:52.965314 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d547eba8-9973-475f-aac6-6a4c43961eec None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:52.965661 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d547eba8-9973-475f-aac6-6a4c43961eec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:52.985339 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-d547eba8-9973-475f-aac6-6a4c43961eec None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:45:52.986528 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-d547eba8-9973-475f-aac6-6a4c43961eec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:45:52.994372 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1054/2128] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:45:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:45:52.998634 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f180b13d-d7ca-4910-9c20-bbf4a6c62537 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:53.008215 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f180b13d-d7ca-4910-9c20-bbf4a6c62537 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:53.013009 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b5e56e-32a8-420b-a4be-c0d32018cb7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:53.013707 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b5e56e-32a8-420b-a4be-c0d32018cb7f admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:53.013991 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b5e56e-32a8-420b-a4be-c0d32018cb7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:53.014266 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b5e56e-32a8-420b-a4be-c0d32018cb7f admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:53.015040 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b5e56e-32a8-420b-a4be-c0d32018cb7f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:45:53.015345 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b5e56e-32a8-420b-a4be-c0d32018cb7f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:53.016972 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b5e56e-32a8-420b-a4be-c0d32018cb7f admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:53.028259 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1075/2129] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jun 16 17:45:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:45:53.033302 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d547eba8-9973-475f-aac6-6a4c43961eec None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:53.043289 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d547eba8-9973-475f-aac6-6a4c43961eec None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:53.049125 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275fc79e-7684-48f4-900a-0a9b1ddad2fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:53.049820 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275fc79e-7684-48f4-900a-0a9b1ddad2fc admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:53.050108 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275fc79e-7684-48f4-900a-0a9b1ddad2fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:53.050387 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275fc79e-7684-48f4-900a-0a9b1ddad2fc admin admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:53.051086 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275fc79e-7684-48f4-900a-0a9b1ddad2fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:53.053088 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275fc79e-7684-48f4-900a-0a9b1ddad2fc admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:53.083416 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1055/2130] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jun 16 17:45:53 2022] POST /identity/v3/projects => generated 361 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:45:53.084555 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b5e56e-32a8-420b-a4be-c0d32018cb7f admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:53.098144 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b5e56e-32a8-420b-a4be-c0d32018cb7f admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:53.151259 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:45:53.162072 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccacd51-9640-4f6b-8d32-45692a4f4b98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:53.163044 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccacd51-9640-4f6b-8d32-45692a4f4b98 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:53.163451 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccacd51-9640-4f6b-8d32-45692a4f4b98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:53.163981 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccacd51-9640-4f6b-8d32-45692a4f4b98 admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:53.165122 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccacd51-9640-4f6b-8d32-45692a4f4b98 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:53.167093 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccacd51-9640-4f6b-8d32-45692a4f4b98 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:53.193267 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1076/2131] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jun 16 17:45:53 2022] POST /identity/v3/users => generated 281 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:45:53.194782 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275fc79e-7684-48f4-900a-0a9b1ddad2fc admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:53.207388 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275fc79e-7684-48f4-900a-0a9b1ddad2fc admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:53.212348 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7e557b-860b-41a2-a6ca-8c51edbca612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:53.213102 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7e557b-860b-41a2-a6ca-8c51edbca612 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:53.213685 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7e557b-860b-41a2-a6ca-8c51edbca612 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:53.215275 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7e557b-860b-41a2-a6ca-8c51edbca612 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:53.215275 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7e557b-860b-41a2-a6ca-8c51edbca612 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:53.216312 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7e557b-860b-41a2-a6ca-8c51edbca612 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:53.227817 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1056/2132] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:45:53 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 17:45:53.232048 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccacd51-9640-4f6b-8d32-45692a4f4b98 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:53.242727 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccacd51-9640-4f6b-8d32-45692a4f4b98 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:53.252500 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88079d5-8aef-4d37-b689-a6e5640aa472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:53.252500 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88079d5-8aef-4d37-b689-a6e5640aa472 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:53.252500 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88079d5-8aef-4d37-b689-a6e5640aa472 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:53.252500 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88079d5-8aef-4d37-b689-a6e5640aa472 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:53.252853 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88079d5-8aef-4d37-b689-a6e5640aa472 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:53.252853 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88079d5-8aef-4d37-b689-a6e5640aa472 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:53.261957 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1077/2133] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:45:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:45:53.266511 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7e557b-860b-41a2-a6ca-8c51edbca612 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:53.276557 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7e557b-860b-41a2-a6ca-8c51edbca612 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:53.281911 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745fb24f-e2b7-4244-bc70-f17c73349e06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:53.282783 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745fb24f-e2b7-4244-bc70-f17c73349e06 admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:53.283069 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745fb24f-e2b7-4244-bc70-f17c73349e06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:53.283350 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745fb24f-e2b7-4244-bc70-f17c73349e06 admin admin] PATH_INFO: `/v3/projects/4d633241bcfb44468494ece558ac3d10/users/43b0221ea1c04689856ed312a6334cb2/roles/0be2075946f54c8485f1c275f746478a` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:53.297669 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745fb24f-e2b7-4244-bc70-f17c73349e06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d633241bcfb44468494ece558ac3d10, user_id=43b0221ea1c04689856ed312a6334cb2, role_id=0be2075946f54c8485f1c275f746478a)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:53.300337 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745fb24f-e2b7-4244-bc70-f17c73349e06 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:53.313920 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1057/2134] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:45:53 2022] PUT /identity/v3/projects/4d633241bcfb44468494ece558ac3d10/users/43b0221ea1c04689856ed312a6334cb2/roles/0be2075946f54c8485f1c275f746478a => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:45:53.324591 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85882758-fd09-479e-8e88-fe82a7325aeb None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:53.324888 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85882758-fd09-479e-8e88-fe82a7325aeb None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:53.325177 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85882758-fd09-479e-8e88-fe82a7325aeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:53.357728 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-85882758-fd09-479e-8e88-fe82a7325aeb None None] MFA Rules not processed for user `43b0221ea1c04689856ed312a6334cb2`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:45:53.359058 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-85882758-fd09-479e-8e88-fe82a7325aeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:45:53.422200 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1078/2135] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:45:53 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:45:53.455039 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745fb24f-e2b7-4244-bc70-f17c73349e06 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:53.457977 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-745fb24f-e2b7-4244-bc70-f17c73349e06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:45:53.519897 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745fb24f-e2b7-4244-bc70-f17c73349e06 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:53.523424 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7f5b10-bf1c-4ac1-aa86-792c4144993b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24d5e2637b7043e08313909d5b06e329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:53.524061 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7f5b10-bf1c-4ac1-aa86-792c4144993b service cinder] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:53.524348 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7f5b10-bf1c-4ac1-aa86-792c4144993b service cinder] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:53.524655 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7f5b10-bf1c-4ac1-aa86-792c4144993b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:53.533736 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7f5b10-bf1c-4ac1-aa86-792c4144993b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:53.535150 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7f5b10-bf1c-4ac1-aa86-792c4144993b service cinder] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:53.543835 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1058/2136] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:45:53 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:45:53.558182 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85882758-fd09-479e-8e88-fe82a7325aeb None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:53.620488 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85882758-fd09-479e-8e88-fe82a7325aeb None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:53.625896 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99165ab0-3238-416a-8089-1b7d96c8011c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:53.626769 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99165ab0-3238-416a-8089-1b7d96c8011c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:53.627066 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99165ab0-3238-416a-8089-1b7d96c8011c admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:53.627339 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99165ab0-3238-416a-8089-1b7d96c8011c admin admin] PATH_INFO: `/v3/users/43b0221ea1c04689856ed312a6334cb2` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:53.628566 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99165ab0-3238-416a-8089-1b7d96c8011c admin admin] RBAC: Authorizing `identity:delete_user(user_id=43b0221ea1c04689856ed312a6334cb2)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:53.630476 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99165ab0-3238-416a-8089-1b7d96c8011c admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:53.675326 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-99165ab0-3238-416a-8089-1b7d96c8011c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43b0221ea1c04689856ed312a6334cb2'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:53.687823 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-99165ab0-3238-416a-8089-1b7d96c8011c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43b0221ea1c04689856ed312a6334cb2'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:53.688526 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-99165ab0-3238-416a-8089-1b7d96c8011c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43b0221ea1c04689856ed312a6334cb2'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:53.689787 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:45:53.697960 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-99165ab0-3238-416a-8089-1b7d96c8011c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43b0221ea1c04689856ed312a6334cb2'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:53.706354 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-99165ab0-3238-416a-8089-1b7d96c8011c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43b0221ea1c04689856ed312a6334cb2'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:53.712499 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1079/2137] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jun 16 17:45:53 2022] DELETE /identity/v3/users/43b0221ea1c04689856ed312a6334cb2 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:45:53.717540 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8703680-644a-4d7e-97b9-a69cc79e7571 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:53.717832 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8703680-644a-4d7e-97b9-a69cc79e7571 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:53.718114 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8703680-644a-4d7e-97b9-a69cc79e7571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:53.735184 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-d8703680-644a-4d7e-97b9-a69cc79e7571 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:45:53.736392 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-d8703680-644a-4d7e-97b9-a69cc79e7571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:45:53.783040 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:DELETE=7,keystone:SELECT=90,keystone:INSERT=3 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:45:53.801233 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1059/2138] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:45:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:45:53.811766 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99165ab0-3238-416a-8089-1b7d96c8011c admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:53.813337 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-99165ab0-3238-416a-8089-1b7d96c8011c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:45:53.870134 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99165ab0-3238-416a-8089-1b7d96c8011c admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:53.874679 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fa7c11-313a-4ed2-9bbd-c23435fccdcc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:53.875327 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fa7c11-313a-4ed2-9bbd-c23435fccdcc service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:53.875612 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fa7c11-313a-4ed2-9bbd-c23435fccdcc service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:53.875890 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fa7c11-313a-4ed2-9bbd-c23435fccdcc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:53.885442 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fa7c11-313a-4ed2-9bbd-c23435fccdcc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:53.886890 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fa7c11-313a-4ed2-9bbd-c23435fccdcc service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:53.897998 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1080/2139] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:45:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:45:54.170076 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8703680-644a-4d7e-97b9-a69cc79e7571 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:54.171581 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-d8703680-644a-4d7e-97b9-a69cc79e7571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:45:54.183882 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8703680-644a-4d7e-97b9-a69cc79e7571 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:54.189790 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bce743e-c15a-4a46-9a39-831badea5036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:54.190668 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bce743e-c15a-4a46-9a39-831badea5036 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:54.190959 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bce743e-c15a-4a46-9a39-831badea5036 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:54.191241 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bce743e-c15a-4a46-9a39-831badea5036 admin admin] PATH_INFO: `/v3/projects/4d633241bcfb44468494ece558ac3d10` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:54.192965 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bce743e-c15a-4a46-9a39-831badea5036 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d633241bcfb44468494ece558ac3d10)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:54.195106 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bce743e-c15a-4a46-9a39-831badea5036 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:54.242082 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-9bce743e-c15a-4a46-9a39-831badea5036 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d633241bcfb44468494ece558ac3d10', 'request_id': 'req-9bce743e-c15a-4a46-9a39-831badea5036'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:54.249589 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-9bce743e-c15a-4a46-9a39-831badea5036 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d633241bcfb44468494ece558ac3d10', 'request_id': 'req-9bce743e-c15a-4a46-9a39-831badea5036'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:54.255493 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-9bce743e-c15a-4a46-9a39-831badea5036 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d633241bcfb44468494ece558ac3d10', 'request_id': 'req-9bce743e-c15a-4a46-9a39-831badea5036'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:54.269359 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-9bce743e-c15a-4a46-9a39-831badea5036 admin admin] The token cache is being invalidate because project 4d633241bcfb44468494ece558ac3d10 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80357) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:45:54.269637 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-9bce743e-c15a-4a46-9a39-831badea5036 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:54.270289 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-9bce743e-c15a-4a46-9a39-831badea5036 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:54.272014 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1060/2140] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jun 16 17:45:54 2022] DELETE /identity/v3/projects/4d633241bcfb44468494ece558ac3d10 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:45:54.276810 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c503a8d-282d-45ca-b677-c4431aad591a None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:54.277288 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c503a8d-282d-45ca-b677-c4431aad591a None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:54.277376 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c503a8d-282d-45ca-b677-c4431aad591a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:54.296291 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-4c503a8d-282d-45ca-b677-c4431aad591a None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:45:54.297529 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-4c503a8d-282d-45ca-b677-c4431aad591a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:45:54.358443 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1081/2141] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:45:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:45:54.366215 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197233ed-b481-4d55-9d2b-551f7f480f3e None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:54.366505 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197233ed-b481-4d55-9d2b-551f7f480f3e None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:54.366783 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197233ed-b481-4d55-9d2b-551f7f480f3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:54.385126 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-197233ed-b481-4d55-9d2b-551f7f480f3e None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:45:54.386326 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-197233ed-b481-4d55-9d2b-551f7f480f3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:45:54.395828 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1061/2142] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:45:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:45:54.399963 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c503a8d-282d-45ca-b677-c4431aad591a None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:54.409917 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c503a8d-282d-45ca-b677-c4431aad591a None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:54.414704 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed353b67-51a8-44f4-9a05-f3a55382730a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:54.415360 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed353b67-51a8-44f4-9a05-f3a55382730a admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:54.415644 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed353b67-51a8-44f4-9a05-f3a55382730a admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:54.415922 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed353b67-51a8-44f4-9a05-f3a55382730a admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:54.416682 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed353b67-51a8-44f4-9a05-f3a55382730a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:45:54.416992 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed353b67-51a8-44f4-9a05-f3a55382730a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:54.418575 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed353b67-51a8-44f4-9a05-f3a55382730a admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:54.430538 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1082/2143] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jun 16 17:45:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:45:54.434852 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197233ed-b481-4d55-9d2b-551f7f480f3e None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:54.444779 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197233ed-b481-4d55-9d2b-551f7f480f3e None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:54.449465 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42f74e5-0987-47e9-883c-9f073028e099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:54.450113 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42f74e5-0987-47e9-883c-9f073028e099 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:54.450424 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42f74e5-0987-47e9-883c-9f073028e099 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:54.450723 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42f74e5-0987-47e9-883c-9f073028e099 admin admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:54.452065 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42f74e5-0987-47e9-883c-9f073028e099 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:54.456766 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42f74e5-0987-47e9-883c-9f073028e099 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:54.484972 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1062/2144] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jun 16 17:45:54 2022] POST /identity/v3/projects => generated 375 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:45:54.489136 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed353b67-51a8-44f4-9a05-f3a55382730a admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:54.499716 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed353b67-51a8-44f4-9a05-f3a55382730a admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:54.554649 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da86e078-8ada-474d-81cf-fb8dded749a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:54.555247 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da86e078-8ada-474d-81cf-fb8dded749a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:54.555533 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da86e078-8ada-474d-81cf-fb8dded749a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:54.555812 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da86e078-8ada-474d-81cf-fb8dded749a0 admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:54.556545 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da86e078-8ada-474d-81cf-fb8dded749a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:54.558456 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da86e078-8ada-474d-81cf-fb8dded749a0 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:54.580326 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1083/2145] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jun 16 17:45:54 2022] POST /identity/v3/users => generated 288 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:45:54.584659 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42f74e5-0987-47e9-883c-9f073028e099 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:54.595386 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42f74e5-0987-47e9-883c-9f073028e099 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:54.600297 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cd7f00-ecc1-4e03-85c6-1e37a4489775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:54.600991 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cd7f00-ecc1-4e03-85c6-1e37a4489775 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:54.601275 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cd7f00-ecc1-4e03-85c6-1e37a4489775 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:54.601552 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cd7f00-ecc1-4e03-85c6-1e37a4489775 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:54.602389 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cd7f00-ecc1-4e03-85c6-1e37a4489775 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:54.603999 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cd7f00-ecc1-4e03-85c6-1e37a4489775 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:54.616075 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1063/2146] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:45:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:45:54.619919 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da86e078-8ada-474d-81cf-fb8dded749a0 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:54.625736 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:45:54.632216 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da86e078-8ada-474d-81cf-fb8dded749a0 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:54.638239 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652911af-1aea-432f-84bf-8d8207b76edc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:54.639523 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652911af-1aea-432f-84bf-8d8207b76edc admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:54.639807 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652911af-1aea-432f-84bf-8d8207b76edc admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:54.640076 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652911af-1aea-432f-84bf-8d8207b76edc admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:54.640825 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652911af-1aea-432f-84bf-8d8207b76edc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:54.642429 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652911af-1aea-432f-84bf-8d8207b76edc admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:54.652222 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1084/2147] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:45:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:45:54.656361 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cd7f00-ecc1-4e03-85c6-1e37a4489775 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:54.666448 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cd7f00-ecc1-4e03-85c6-1e37a4489775 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:54.671484 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e035d980-05ae-408c-a4d3-6cbcd6461ddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:54.672351 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e035d980-05ae-408c-a4d3-6cbcd6461ddd admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:54.672678 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e035d980-05ae-408c-a4d3-6cbcd6461ddd admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:54.672952 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e035d980-05ae-408c-a4d3-6cbcd6461ddd admin admin] PATH_INFO: `/v3/projects/5673381c02e74786872844cf40f7410a/users/d2e119d9109843ac8eb5ac447749a4fc/roles/0be2075946f54c8485f1c275f746478a` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:54.687760 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e035d980-05ae-408c-a4d3-6cbcd6461ddd admin admin] RBAC: Authorizing `identity:create_grant(project_id=5673381c02e74786872844cf40f7410a, user_id=d2e119d9109843ac8eb5ac447749a4fc, role_id=0be2075946f54c8485f1c275f746478a)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:54.690436 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e035d980-05ae-408c-a4d3-6cbcd6461ddd admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:54.703598 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1064/2148] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:45:54 2022] PUT /identity/v3/projects/5673381c02e74786872844cf40f7410a/users/d2e119d9109843ac8eb5ac447749a4fc/roles/0be2075946f54c8485f1c275f746478a => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:45:54.714908 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136cdaeb-07c0-4697-9639-68f7bbb50b83 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:54.715212 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136cdaeb-07c0-4697-9639-68f7bbb50b83 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:54.715486 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136cdaeb-07c0-4697-9639-68f7bbb50b83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:54.743273 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-136cdaeb-07c0-4697-9639-68f7bbb50b83 None None] MFA Rules not processed for user `d2e119d9109843ac8eb5ac447749a4fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:45:54.744536 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-136cdaeb-07c0-4697-9639-68f7bbb50b83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:45:54.819704 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1085/2149] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:45:54 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:45:54.824079 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e035d980-05ae-408c-a4d3-6cbcd6461ddd admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:54.887242 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e035d980-05ae-408c-a4d3-6cbcd6461ddd admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:54.892183 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df856007-331c-40ea-9069-8d7b5cd9bb44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:54.892938 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df856007-331c-40ea-9069-8d7b5cd9bb44 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:54.893226 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df856007-331c-40ea-9069-8d7b5cd9bb44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:54.893506 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df856007-331c-40ea-9069-8d7b5cd9bb44 admin admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:54.894206 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df856007-331c-40ea-9069-8d7b5cd9bb44 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:54.896114 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df856007-331c-40ea-9069-8d7b5cd9bb44 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:54.909228 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:45:54.924110 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1065/2150] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jun 16 17:45:54 2022] POST /identity/v3/projects => generated 375 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:45:54.928075 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136cdaeb-07c0-4697-9639-68f7bbb50b83 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:54.939868 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136cdaeb-07c0-4697-9639-68f7bbb50b83 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:54.996354 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e467682f-4e5d-41c5-b6cb-77ce4918be96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:54.997138 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e467682f-4e5d-41c5-b6cb-77ce4918be96 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:54.997426 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e467682f-4e5d-41c5-b6cb-77ce4918be96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:54.998304 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e467682f-4e5d-41c5-b6cb-77ce4918be96 admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:54.999010 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e467682f-4e5d-41c5-b6cb-77ce4918be96 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:55.000961 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e467682f-4e5d-41c5-b6cb-77ce4918be96 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:55.024629 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1086/2151] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jun 16 17:45:54 2022] POST /identity/v3/users => generated 294 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:45:55.025451 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df856007-331c-40ea-9069-8d7b5cd9bb44 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:55.036850 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df856007-331c-40ea-9069-8d7b5cd9bb44 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:55.041719 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eb184b-9bf3-4c3b-b101-06c13faecf42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:55.042386 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eb184b-9bf3-4c3b-b101-06c13faecf42 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:55.042667 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eb184b-9bf3-4c3b-b101-06c13faecf42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:55.042939 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eb184b-9bf3-4c3b-b101-06c13faecf42 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:55.043666 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49eb184b-9bf3-4c3b-b101-06c13faecf42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:55.045984 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49eb184b-9bf3-4c3b-b101-06c13faecf42 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:55.056579 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1066/2152] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:45:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 17:45:55.061599 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e467682f-4e5d-41c5-b6cb-77ce4918be96 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:55.071885 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e467682f-4e5d-41c5-b6cb-77ce4918be96 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:55.076723 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52234211-b9f3-4009-915f-a5a7fab443f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:55.077594 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52234211-b9f3-4009-915f-a5a7fab443f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:55.077944 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52234211-b9f3-4009-915f-a5a7fab443f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:55.078226 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52234211-b9f3-4009-915f-a5a7fab443f6 admin admin] PATH_INFO: `/v3/projects/18bcaba2f9794d2f988e890fd6a07d18/users/c08b2e28c2bb427e8a5d011230dfb91d/roles/3ca2dcbbd29d4a31a3ab78f2299bc239` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:55.100502 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52234211-b9f3-4009-915f-a5a7fab443f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=18bcaba2f9794d2f988e890fd6a07d18, user_id=c08b2e28c2bb427e8a5d011230dfb91d, role_id=3ca2dcbbd29d4a31a3ab78f2299bc239)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:55.104763 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52234211-b9f3-4009-915f-a5a7fab443f6 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:55.121829 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1087/2153] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:45:55 2022] PUT /identity/v3/projects/18bcaba2f9794d2f988e890fd6a07d18/users/c08b2e28c2bb427e8a5d011230dfb91d/roles/3ca2dcbbd29d4a31a3ab78f2299bc239 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:45:55.125983 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eb184b-9bf3-4c3b-b101-06c13faecf42 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:55.138229 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eb184b-9bf3-4c3b-b101-06c13faecf42 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:55.199653 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a381de8-a969-4d31-8e14-a879a1979948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:55.200326 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a381de8-a969-4d31-8e14-a879a1979948 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:55.200634 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a381de8-a969-4d31-8e14-a879a1979948 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:55.201125 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a381de8-a969-4d31-8e14-a879a1979948 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:55.201616 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a381de8-a969-4d31-8e14-a879a1979948 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:55.203205 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a381de8-a969-4d31-8e14-a879a1979948 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:55.213651 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1067/2154] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:45:55 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:45:55.218382 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52234211-b9f3-4009-915f-a5a7fab443f6 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:55.228743 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52234211-b9f3-4009-915f-a5a7fab443f6 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:55.234090 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df1ffbe-1c64-4713-b330-a07e48b7a509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:55.234968 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df1ffbe-1c64-4713-b330-a07e48b7a509 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:55.235247 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df1ffbe-1c64-4713-b330-a07e48b7a509 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:55.235525 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df1ffbe-1c64-4713-b330-a07e48b7a509 admin admin] PATH_INFO: `/v3/projects/18bcaba2f9794d2f988e890fd6a07d18/users/c08b2e28c2bb427e8a5d011230dfb91d/roles/3ca2dcbbd29d4a31a3ab78f2299bc239` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:55.237513 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df1ffbe-1c64-4713-b330-a07e48b7a509 admin admin] RBAC: Authorizing `identity:create_grant(project_id=18bcaba2f9794d2f988e890fd6a07d18, user_id=c08b2e28c2bb427e8a5d011230dfb91d, role_id=3ca2dcbbd29d4a31a3ab78f2299bc239)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:55.240120 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df1ffbe-1c64-4713-b330-a07e48b7a509 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:55.250839 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1088/2155] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:45:55 2022] PUT /identity/v3/projects/18bcaba2f9794d2f988e890fd6a07d18/users/c08b2e28c2bb427e8a5d011230dfb91d/roles/3ca2dcbbd29d4a31a3ab78f2299bc239 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:45:55.261899 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceac16d-fbd0-4970-aa96-bf7fb85f3677 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:55.261899 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceac16d-fbd0-4970-aa96-bf7fb85f3677 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:55.262084 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceac16d-fbd0-4970-aa96-bf7fb85f3677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:55.289688 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-7ceac16d-fbd0-4970-aa96-bf7fb85f3677 None None] MFA Rules not processed for user `c08b2e28c2bb427e8a5d011230dfb91d`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:45:55.290402 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-7ceac16d-fbd0-4970-aa96-bf7fb85f3677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:45:55.377608 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1068/2156] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:45:55 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:45:55.388554 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df1ffbe-1c64-4713-b330-a07e48b7a509 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:55.391060 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-9df1ffbe-1c64-4713-b330-a07e48b7a509 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:45:55.430000 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:45:55.457488 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df1ffbe-1c64-4713-b330-a07e48b7a509 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:55.461240 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9681a734-0e7e-4f60-ab55-4c7b4d8aaedb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24d5e2637b7043e08313909d5b06e329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:55.461886 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9681a734-0e7e-4f60-ab55-4c7b4d8aaedb service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:55.462172 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9681a734-0e7e-4f60-ab55-4c7b4d8aaedb service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:55.462448 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9681a734-0e7e-4f60-ab55-4c7b4d8aaedb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:55.520737 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9681a734-0e7e-4f60-ab55-4c7b4d8aaedb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:55.522197 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9681a734-0e7e-4f60-ab55-4c7b4d8aaedb service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:55.531269 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1089/2157] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:45:55 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:45:58.050768 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ceac16d-fbd0-4970-aa96-bf7fb85f3677 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:58.114681 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:45:58.157823 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ceac16d-fbd0-4970-aa96-bf7fb85f3677 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:58.162303 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f312260-577f-48e0-a8e9-eb256e8dbdc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:58.163174 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f312260-577f-48e0-a8e9-eb256e8dbdc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:58.163450 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f312260-577f-48e0-a8e9-eb256e8dbdc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:58.163721 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f312260-577f-48e0-a8e9-eb256e8dbdc7 admin admin] PATH_INFO: `/v3/users/d2e119d9109843ac8eb5ac447749a4fc` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:58.165029 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f312260-577f-48e0-a8e9-eb256e8dbdc7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2e119d9109843ac8eb5ac447749a4fc)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:58.167093 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f312260-577f-48e0-a8e9-eb256e8dbdc7 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:58.214484 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-1f312260-577f-48e0-a8e9-eb256e8dbdc7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2e119d9109843ac8eb5ac447749a4fc'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:58.215127 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-1f312260-577f-48e0-a8e9-eb256e8dbdc7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2e119d9109843ac8eb5ac447749a4fc'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:58.222704 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-1f312260-577f-48e0-a8e9-eb256e8dbdc7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2e119d9109843ac8eb5ac447749a4fc'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:58.223326 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-1f312260-577f-48e0-a8e9-eb256e8dbdc7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2e119d9109843ac8eb5ac447749a4fc'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:58.235035 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-1f312260-577f-48e0-a8e9-eb256e8dbdc7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2e119d9109843ac8eb5ac447749a4fc'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:58.248408 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1069/2158] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jun 16 17:45:58 2022] DELETE /identity/v3/users/d2e119d9109843ac8eb5ac447749a4fc => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:45:58.253288 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d928502d-8dd7-4521-b93c-d170b34fa7c9 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:58.253767 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d928502d-8dd7-4521-b93c-d170b34fa7c9 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:58.253861 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d928502d-8dd7-4521-b93c-d170b34fa7c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:58.303474 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-d928502d-8dd7-4521-b93c-d170b34fa7c9 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:45:58.304720 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-d928502d-8dd7-4521-b93c-d170b34fa7c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:45:58.364007 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1090/2159] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:45:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:45:58.373619 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f312260-577f-48e0-a8e9-eb256e8dbdc7 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:58.375120 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-1f312260-577f-48e0-a8e9-eb256e8dbdc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:45:58.470681 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f312260-577f-48e0-a8e9-eb256e8dbdc7 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:58.474415 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5027876-f924-470c-b82e-df59e4962b53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:58.475066 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5027876-f924-470c-b82e-df59e4962b53 service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:58.475347 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5027876-f924-470c-b82e-df59e4962b53 service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:58.475623 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5027876-f924-470c-b82e-df59e4962b53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:58.485424 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5027876-f924-470c-b82e-df59e4962b53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:58.486854 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5027876-f924-470c-b82e-df59e4962b53 service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:58.496608 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1070/2160] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:45:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 17:45:58.834446 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d928502d-8dd7-4521-b93c-d170b34fa7c9 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:58.843332 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-d928502d-8dd7-4521-b93c-d170b34fa7c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:45:58.858264 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d928502d-8dd7-4521-b93c-d170b34fa7c9 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:58.874448 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2857f6-f53d-433c-bdd0-5a9424c850bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:58.875808 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2857f6-f53d-433c-bdd0-5a9424c850bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:58.876329 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2857f6-f53d-433c-bdd0-5a9424c850bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:58.877568 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2857f6-f53d-433c-bdd0-5a9424c850bf admin admin] PATH_INFO: `/v3/projects/5673381c02e74786872844cf40f7410a` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:58.878526 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2857f6-f53d-433c-bdd0-5a9424c850bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=5673381c02e74786872844cf40f7410a)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:58.881667 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2857f6-f53d-433c-bdd0-5a9424c850bf admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:58.945362 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-1a2857f6-f53d-433c-bdd0-5a9424c850bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5673381c02e74786872844cf40f7410a', 'request_id': 'req-1a2857f6-f53d-433c-bdd0-5a9424c850bf'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:58.953652 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-1a2857f6-f53d-433c-bdd0-5a9424c850bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5673381c02e74786872844cf40f7410a', 'request_id': 'req-1a2857f6-f53d-433c-bdd0-5a9424c850bf'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:58.954701 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:45:58.960453 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-1a2857f6-f53d-433c-bdd0-5a9424c850bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5673381c02e74786872844cf40f7410a', 'request_id': 'req-1a2857f6-f53d-433c-bdd0-5a9424c850bf'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:58.972544 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-1a2857f6-f53d-433c-bdd0-5a9424c850bf admin admin] The token cache is being invalidate because project 5673381c02e74786872844cf40f7410a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:45:58.972831 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-1a2857f6-f53d-433c-bdd0-5a9424c850bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:58.973500 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-1a2857f6-f53d-433c-bdd0-5a9424c850bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:58.975338 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1091/2161] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jun 16 17:45:58 2022] DELETE /identity/v3/projects/5673381c02e74786872844cf40f7410a => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:45:58.979420 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5027876-f924-470c-b82e-df59e4962b53 service neutron] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:58.990442 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5027876-f924-470c-b82e-df59e4962b53 service neutron] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:58.995138 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:45:59.001920 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2857f6-f53d-433c-bdd0-5a9424c850bf admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:59.003601 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-1a2857f6-f53d-433c-bdd0-5a9424c850bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:45:59.051897 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a115c399-5e2b-4ed9-b19e-599c1f91fa00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:59.052838 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a115c399-5e2b-4ed9-b19e-599c1f91fa00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:59.052999 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a115c399-5e2b-4ed9-b19e-599c1f91fa00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:59.053292 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a115c399-5e2b-4ed9-b19e-599c1f91fa00 admin admin] PATH_INFO: `/v3/users/c08b2e28c2bb427e8a5d011230dfb91d` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:59.054476 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a115c399-5e2b-4ed9-b19e-599c1f91fa00 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c08b2e28c2bb427e8a5d011230dfb91d)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:59.056410 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a115c399-5e2b-4ed9-b19e-599c1f91fa00 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:59.063400 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2857f6-f53d-433c-bdd0-5a9424c850bf admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:59.067067 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2204a5b-3e71-4c51-9f94-0d88affff759 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24d5e2637b7043e08313909d5b06e329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:59.067731 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2204a5b-3e71-4c51-9f94-0d88affff759 service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:59.067998 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2204a5b-3e71-4c51-9f94-0d88affff759 service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:59.068274 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2204a5b-3e71-4c51-9f94-0d88affff759 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:59.070159 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-d2204a5b-3e71-4c51-9f94-0d88affff759 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:45:59.103881 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-a115c399-5e2b-4ed9-b19e-599c1f91fa00 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c08b2e28c2bb427e8a5d011230dfb91d'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:59.104579 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-a115c399-5e2b-4ed9-b19e-599c1f91fa00 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c08b2e28c2bb427e8a5d011230dfb91d'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:59.113707 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-a115c399-5e2b-4ed9-b19e-599c1f91fa00 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c08b2e28c2bb427e8a5d011230dfb91d'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:59.114563 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-a115c399-5e2b-4ed9-b19e-599c1f91fa00 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c08b2e28c2bb427e8a5d011230dfb91d'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:59.128320 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-a115c399-5e2b-4ed9-b19e-599c1f91fa00 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c08b2e28c2bb427e8a5d011230dfb91d'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:59.134074 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2204a5b-3e71-4c51-9f94-0d88affff759 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:59.135560 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2204a5b-3e71-4c51-9f94-0d88affff759 service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:59.142075 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1071/2162] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jun 16 17:45:58 2022] DELETE /identity/v3/users/c08b2e28c2bb427e8a5d011230dfb91d => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:45:59.145580 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1092/2163] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:45:59 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 17:45:59.468919 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a115c399-5e2b-4ed9-b19e-599c1f91fa00 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:59.470684 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-a115c399-5e2b-4ed9-b19e-599c1f91fa00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:45:59.548703 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a115c399-5e2b-4ed9-b19e-599c1f91fa00 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:59.553704 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05c7916-15e2-4a77-9e68-62aa2fb92e3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:59.554734 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05c7916-15e2-4a77-9e68-62aa2fb92e3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:59.555022 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05c7916-15e2-4a77-9e68-62aa2fb92e3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:59.555300 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05c7916-15e2-4a77-9e68-62aa2fb92e3a admin admin] PATH_INFO: `/v3/projects/18bcaba2f9794d2f988e890fd6a07d18` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:59.556609 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05c7916-15e2-4a77-9e68-62aa2fb92e3a admin admin] RBAC: Authorizing `identity:delete_project(project_id=18bcaba2f9794d2f988e890fd6a07d18)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:59.558492 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05c7916-15e2-4a77-9e68-62aa2fb92e3a admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:59.613619 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:45:59.635698 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-b05c7916-15e2-4a77-9e68-62aa2fb92e3a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18bcaba2f9794d2f988e890fd6a07d18', 'request_id': 'req-b05c7916-15e2-4a77-9e68-62aa2fb92e3a'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:59.645288 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-b05c7916-15e2-4a77-9e68-62aa2fb92e3a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18bcaba2f9794d2f988e890fd6a07d18', 'request_id': 'req-b05c7916-15e2-4a77-9e68-62aa2fb92e3a'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:59.652912 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-b05c7916-15e2-4a77-9e68-62aa2fb92e3a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18bcaba2f9794d2f988e890fd6a07d18', 'request_id': 'req-b05c7916-15e2-4a77-9e68-62aa2fb92e3a'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:59.664638 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-b05c7916-15e2-4a77-9e68-62aa2fb92e3a admin admin] The token cache is being invalidate because project 18bcaba2f9794d2f988e890fd6a07d18 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80357) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:45:59.664934 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-b05c7916-15e2-4a77-9e68-62aa2fb92e3a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:59.665614 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-b05c7916-15e2-4a77-9e68-62aa2fb92e3a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:45:59.667377 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1072/2164] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jun 16 17:45:59 2022] DELETE /identity/v3/projects/18bcaba2f9794d2f988e890fd6a07d18 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:45:59.672721 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e17bf3-df35-4b2f-bbe9-df1c1139a6a9 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:59.673233 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e17bf3-df35-4b2f-bbe9-df1c1139a6a9 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:59.673321 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e17bf3-df35-4b2f-bbe9-df1c1139a6a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:59.713305 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-d5e17bf3-df35-4b2f-bbe9-df1c1139a6a9 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:45:59.713305 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-d5e17bf3-df35-4b2f-bbe9-df1c1139a6a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:45:59.777518 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1093/2165] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:45:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:45:59.786692 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1609554a-9458-4ffb-b0b2-ffe52feddb81 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:59.787081 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1609554a-9458-4ffb-b0b2-ffe52feddb81 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:59.787345 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1609554a-9458-4ffb-b0b2-ffe52feddb81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:59.806478 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-1609554a-9458-4ffb-b0b2-ffe52feddb81 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:45:59.807682 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-1609554a-9458-4ffb-b0b2-ffe52feddb81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:45:59.817848 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1073/2166] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:45:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:45:59.824201 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e17bf3-df35-4b2f-bbe9-df1c1139a6a9 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:59.835112 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e17bf3-df35-4b2f-bbe9-df1c1139a6a9 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:59.839818 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559c851e-621a-4be2-b51e-e8a73c8107d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:59.840509 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559c851e-621a-4be2-b51e-e8a73c8107d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:59.840793 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559c851e-621a-4be2-b51e-e8a73c8107d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:59.841075 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559c851e-621a-4be2-b51e-e8a73c8107d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:59.841835 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559c851e-621a-4be2-b51e-e8a73c8107d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:45:59.842168 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559c851e-621a-4be2-b51e-e8a73c8107d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:59.843768 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559c851e-621a-4be2-b51e-e8a73c8107d2 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:59.856115 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1094/2167] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jun 16 17:45:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:45:59.861944 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1609554a-9458-4ffb-b0b2-ffe52feddb81 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:59.880056 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1609554a-9458-4ffb-b0b2-ffe52feddb81 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:59.885057 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a588c637-95ff-4897-bf0c-abfb34fca530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:45:59.885720 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a588c637-95ff-4897-bf0c-abfb34fca530 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:59.886003 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a588c637-95ff-4897-bf0c-abfb34fca530 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:59.886275 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a588c637-95ff-4897-bf0c-abfb34fca530 admin admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:59.886967 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a588c637-95ff-4897-bf0c-abfb34fca530 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:45:59.889549 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a588c637-95ff-4897-bf0c-abfb34fca530 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:45:59.916433 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1074/2168] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jun 16 17:45:59 2022] POST /identity/v3/projects => generated 389 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:45:59.918772 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559c851e-621a-4be2-b51e-e8a73c8107d2 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:45:59.929548 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559c851e-621a-4be2-b51e-e8a73c8107d2 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:45:59.941744 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:00.021104 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3db27c-6eca-41f2-96e8-2044e6072039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:00.021910 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3db27c-6eca-41f2-96e8-2044e6072039 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:00.022269 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3db27c-6eca-41f2-96e8-2044e6072039 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:00.022632 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3db27c-6eca-41f2-96e8-2044e6072039 admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:00.023702 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3db27c-6eca-41f2-96e8-2044e6072039 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:00.026188 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3db27c-6eca-41f2-96e8-2044e6072039 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:00.049941 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1095/2169] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jun 16 17:45:59 2022] POST /identity/v3/users => generated 295 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:46:00.054053 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a588c637-95ff-4897-bf0c-abfb34fca530 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:00.072893 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a588c637-95ff-4897-bf0c-abfb34fca530 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:00.078613 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a775bc7-8b7b-491d-84c0-dfdc4d2892c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:00.079386 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a775bc7-8b7b-491d-84c0-dfdc4d2892c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:00.079674 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a775bc7-8b7b-491d-84c0-dfdc4d2892c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:00.080132 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a775bc7-8b7b-491d-84c0-dfdc4d2892c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:00.080791 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a775bc7-8b7b-491d-84c0-dfdc4d2892c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:00.082354 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a775bc7-8b7b-491d-84c0-dfdc4d2892c8 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:00.094069 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1075/2170] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:46:00 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:46:00.098078 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3db27c-6eca-41f2-96e8-2044e6072039 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:00.109038 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3db27c-6eca-41f2-96e8-2044e6072039 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:00.114214 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c72564-16be-46d8-ad14-8e8e53725d7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:00.114898 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c72564-16be-46d8-ad14-8e8e53725d7c admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:00.115165 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c72564-16be-46d8-ad14-8e8e53725d7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:00.115440 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c72564-16be-46d8-ad14-8e8e53725d7c admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:00.116160 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c72564-16be-46d8-ad14-8e8e53725d7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:00.118074 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c72564-16be-46d8-ad14-8e8e53725d7c admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:00.128667 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1096/2171] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:46:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:46:00.132869 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a775bc7-8b7b-491d-84c0-dfdc4d2892c8 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:00.143660 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a775bc7-8b7b-491d-84c0-dfdc4d2892c8 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:00.148359 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20857d1c-0c48-41b1-80f9-102137d5b596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:00.149266 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20857d1c-0c48-41b1-80f9-102137d5b596 admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:00.149548 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20857d1c-0c48-41b1-80f9-102137d5b596 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:00.149823 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20857d1c-0c48-41b1-80f9-102137d5b596 admin admin] PATH_INFO: `/v3/projects/a697b09fbb1845c0b61a090ea1083912/users/b6e833a13903498eb5a49bc0996a4da0/roles/0be2075946f54c8485f1c275f746478a` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:00.165444 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20857d1c-0c48-41b1-80f9-102137d5b596 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a697b09fbb1845c0b61a090ea1083912, user_id=b6e833a13903498eb5a49bc0996a4da0, role_id=0be2075946f54c8485f1c275f746478a)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:00.168102 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20857d1c-0c48-41b1-80f9-102137d5b596 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:00.180957 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1076/2172] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:46:00 2022] PUT /identity/v3/projects/a697b09fbb1845c0b61a090ea1083912/users/b6e833a13903498eb5a49bc0996a4da0/roles/0be2075946f54c8485f1c275f746478a => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:46:00.192240 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba91bf17-795c-4900-85bf-70f4633b2953 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:00.192562 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba91bf17-795c-4900-85bf-70f4633b2953 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:00.192843 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba91bf17-795c-4900-85bf-70f4633b2953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:00.221146 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-ba91bf17-795c-4900-85bf-70f4633b2953 None None] MFA Rules not processed for user `b6e833a13903498eb5a49bc0996a4da0`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:46:00.222547 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-ba91bf17-795c-4900-85bf-70f4633b2953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:46:00.287038 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1097/2173] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:46:00 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:46:00.297771 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20857d1c-0c48-41b1-80f9-102137d5b596 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:00.299445 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-20857d1c-0c48-41b1-80f9-102137d5b596 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:46:00.356262 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20857d1c-0c48-41b1-80f9-102137d5b596 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:00.360042 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb1165c-03a4-498f-966f-bd92afc78ab3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24d5e2637b7043e08313909d5b06e329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:00.360719 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb1165c-03a4-498f-966f-bd92afc78ab3 service cinder] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:00.361008 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb1165c-03a4-498f-966f-bd92afc78ab3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:00.361290 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb1165c-03a4-498f-966f-bd92afc78ab3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:00.369697 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb1165c-03a4-498f-966f-bd92afc78ab3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:00.371124 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb1165c-03a4-498f-966f-bd92afc78ab3 service cinder] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:00.380730 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1077/2174] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:46:00 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:46:01.336231 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba91bf17-795c-4900-85bf-70f4633b2953 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:01.337679 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-ba91bf17-795c-4900-85bf-70f4633b2953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:46:01.417141 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba91bf17-795c-4900-85bf-70f4633b2953 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:01.422599 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6e8332-5faf-4a52-ae72-5508ef55a6ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:01.423561 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6e8332-5faf-4a52-ae72-5508ef55a6ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:01.423927 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6e8332-5faf-4a52-ae72-5508ef55a6ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:01.424209 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6e8332-5faf-4a52-ae72-5508ef55a6ee admin admin] PATH_INFO: `/v3/users/bd919abbe5ce4e07951be9e26c48da56` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:01.425667 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6e8332-5faf-4a52-ae72-5508ef55a6ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd919abbe5ce4e07951be9e26c48da56)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:01.428133 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6e8332-5faf-4a52-ae72-5508ef55a6ee admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:01.450146 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:01.475845 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-da6e8332-5faf-4a52-ae72-5508ef55a6ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd919abbe5ce4e07951be9e26c48da56'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:46:01.489171 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-da6e8332-5faf-4a52-ae72-5508ef55a6ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd919abbe5ce4e07951be9e26c48da56'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:46:01.489859 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-da6e8332-5faf-4a52-ae72-5508ef55a6ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd919abbe5ce4e07951be9e26c48da56'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:46:01.496635 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-da6e8332-5faf-4a52-ae72-5508ef55a6ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd919abbe5ce4e07951be9e26c48da56'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:46:01.503823 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-da6e8332-5faf-4a52-ae72-5508ef55a6ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd919abbe5ce4e07951be9e26c48da56'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:46:01.510136 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1098/2175] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jun 16 17:46:01 2022] DELETE /identity/v3/users/bd919abbe5ce4e07951be9e26c48da56 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:46:01.515277 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dadbe6-1d7c-4fa8-85d5-55266c2c7adc None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:01.515738 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dadbe6-1d7c-4fa8-85d5-55266c2c7adc None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:01.515868 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dadbe6-1d7c-4fa8-85d5-55266c2c7adc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:01.533358 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-e8dadbe6-1d7c-4fa8-85d5-55266c2c7adc None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:46:01.534575 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-e8dadbe6-1d7c-4fa8-85d5-55266c2c7adc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:46:01.584801 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:01.595357 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1078/2176] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:46:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:46:01.604862 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6e8332-5faf-4a52-ae72-5508ef55a6ee admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:01.606438 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-da6e8332-5faf-4a52-ae72-5508ef55a6ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:46:01.668000 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6e8332-5faf-4a52-ae72-5508ef55a6ee admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:01.672481 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a71c59a-093c-491d-a673-a6a499669c7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:01.673158 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a71c59a-093c-491d-a673-a6a499669c7e service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:01.673441 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a71c59a-093c-491d-a673-a6a499669c7e service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:01.673745 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a71c59a-093c-491d-a673-a6a499669c7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:01.683955 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a71c59a-093c-491d-a673-a6a499669c7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:01.686191 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a71c59a-093c-491d-a673-a6a499669c7e service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:01.696337 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1099/2177] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:46:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:46:01.985246 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dadbe6-1d7c-4fa8-85d5-55266c2c7adc None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:01.986688 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-e8dadbe6-1d7c-4fa8-85d5-55266c2c7adc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:46:02.002541 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dadbe6-1d7c-4fa8-85d5-55266c2c7adc None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:02.007921 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d87b97-bb19-4395-a57a-ddb87e7df9fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:02.009077 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d87b97-bb19-4395-a57a-ddb87e7df9fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:02.009360 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d87b97-bb19-4395-a57a-ddb87e7df9fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:02.009639 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d87b97-bb19-4395-a57a-ddb87e7df9fa admin admin] PATH_INFO: `/v3/projects/9e08a72841c641c6ae81856a713ed967` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:02.010948 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d87b97-bb19-4395-a57a-ddb87e7df9fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e08a72841c641c6ae81856a713ed967)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:02.012895 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d87b97-bb19-4395-a57a-ddb87e7df9fa admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:02.059137 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-f7d87b97-bb19-4395-a57a-ddb87e7df9fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e08a72841c641c6ae81856a713ed967', 'request_id': 'req-f7d87b97-bb19-4395-a57a-ddb87e7df9fa'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:46:02.067760 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-f7d87b97-bb19-4395-a57a-ddb87e7df9fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e08a72841c641c6ae81856a713ed967', 'request_id': 'req-f7d87b97-bb19-4395-a57a-ddb87e7df9fa'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:46:02.073793 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-f7d87b97-bb19-4395-a57a-ddb87e7df9fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e08a72841c641c6ae81856a713ed967', 'request_id': 'req-f7d87b97-bb19-4395-a57a-ddb87e7df9fa'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:46:02.085878 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-f7d87b97-bb19-4395-a57a-ddb87e7df9fa admin admin] The token cache is being invalidate because project 9e08a72841c641c6ae81856a713ed967 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80357) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:46:02.086176 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-f7d87b97-bb19-4395-a57a-ddb87e7df9fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:46:02.086813 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-f7d87b97-bb19-4395-a57a-ddb87e7df9fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:46:02.088392 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1079/2178] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jun 16 17:46:01 2022] DELETE /identity/v3/projects/9e08a72841c641c6ae81856a713ed967 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:46:02.092491 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a71c59a-093c-491d-a673-a6a499669c7e service neutron] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:02.102919 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a71c59a-093c-491d-a673-a6a499669c7e service neutron] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:02.160383 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1ba5fa-0719-4d7a-bba0-caba4b63a401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:02.161280 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1ba5fa-0719-4d7a-bba0-caba4b63a401 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:02.161565 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1ba5fa-0719-4d7a-bba0-caba4b63a401 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:02.161838 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1ba5fa-0719-4d7a-bba0-caba4b63a401 admin admin] PATH_INFO: `/v3/users/8f47b5649e484ba1966df56ae838620f` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:02.163017 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1ba5fa-0719-4d7a-bba0-caba4b63a401 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f47b5649e484ba1966df56ae838620f)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:02.164958 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1ba5fa-0719-4d7a-bba0-caba4b63a401 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:02.217158 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:02.220436 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-6e1ba5fa-0719-4d7a-bba0-caba4b63a401 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f47b5649e484ba1966df56ae838620f'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:46:02.224792 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d87b97-bb19-4395-a57a-ddb87e7df9fa admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:02.226271 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-f7d87b97-bb19-4395-a57a-ddb87e7df9fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:46:02.234287 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-6e1ba5fa-0719-4d7a-bba0-caba4b63a401 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f47b5649e484ba1966df56ae838620f'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:46:02.234976 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-6e1ba5fa-0719-4d7a-bba0-caba4b63a401 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f47b5649e484ba1966df56ae838620f'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:46:02.239779 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d87b97-bb19-4395-a57a-ddb87e7df9fa admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:02.241762 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-6e1ba5fa-0719-4d7a-bba0-caba4b63a401 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f47b5649e484ba1966df56ae838620f'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:46:02.244493 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ccdd33-fa6c-40dd-aafa-3720c0978f7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:02.245356 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ccdd33-fa6c-40dd-aafa-3720c0978f7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:02.245637 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ccdd33-fa6c-40dd-aafa-3720c0978f7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:02.245906 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ccdd33-fa6c-40dd-aafa-3720c0978f7a admin admin] PATH_INFO: `/v3/users/1e29ecfb603b4a28aebab4aa5d8f6abe` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:02.247459 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ccdd33-fa6c-40dd-aafa-3720c0978f7a admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e29ecfb603b4a28aebab4aa5d8f6abe)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:02.250186 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ccdd33-fa6c-40dd-aafa-3720c0978f7a admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:02.250811 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-6e1ba5fa-0719-4d7a-bba0-caba4b63a401 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f47b5649e484ba1966df56ae838620f'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:46:02.257485 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1100/2179] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jun 16 17:46:02 2022] DELETE /identity/v3/users/8f47b5649e484ba1966df56ae838620f => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:46:02.305782 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-42ccdd33-fa6c-40dd-aafa-3720c0978f7a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e29ecfb603b4a28aebab4aa5d8f6abe'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:46:02.307172 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-42ccdd33-fa6c-40dd-aafa-3720c0978f7a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e29ecfb603b4a28aebab4aa5d8f6abe'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:46:02.317548 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-42ccdd33-fa6c-40dd-aafa-3720c0978f7a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e29ecfb603b4a28aebab4aa5d8f6abe'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:46:02.318289 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-42ccdd33-fa6c-40dd-aafa-3720c0978f7a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e29ecfb603b4a28aebab4aa5d8f6abe'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:46:02.336184 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-42ccdd33-fa6c-40dd-aafa-3720c0978f7a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e29ecfb603b4a28aebab4aa5d8f6abe'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:46:02.350695 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1080/2180] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jun 16 17:46:02 2022] DELETE /identity/v3/users/1e29ecfb603b4a28aebab4aa5d8f6abe => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:46:02.355147 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c351403-a070-44c7-99ab-b175603e1e2d None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:02.355639 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c351403-a070-44c7-99ab-b175603e1e2d None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:02.355824 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c351403-a070-44c7-99ab-b175603e1e2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:02.376316 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-1c351403-a070-44c7-99ab-b175603e1e2d None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:46:02.377382 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-1c351403-a070-44c7-99ab-b175603e1e2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:46:02.446287 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1101/2181] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:46:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:46:02.456312 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ccdd33-fa6c-40dd-aafa-3720c0978f7a admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:02.457716 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-42ccdd33-fa6c-40dd-aafa-3720c0978f7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:46:02.540473 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ccdd33-fa6c-40dd-aafa-3720c0978f7a admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:02.544220 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c31c602-c161-4e34-a3b0-293de688131d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:02.545381 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c31c602-c161-4e34-a3b0-293de688131d service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:02.545844 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c31c602-c161-4e34-a3b0-293de688131d service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:02.546125 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c31c602-c161-4e34-a3b0-293de688131d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:02.553540 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:02.556508 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c31c602-c161-4e34-a3b0-293de688131d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:02.558643 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c31c602-c161-4e34-a3b0-293de688131d service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:02.572445 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1081/2182] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:46:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:46:02.623408 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c351403-a070-44c7-99ab-b175603e1e2d None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:02.624997 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-1c351403-a070-44c7-99ab-b175603e1e2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:46:02.639342 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c351403-a070-44c7-99ab-b175603e1e2d None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:02.648059 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5754768b-e4f5-4a95-a0be-497d9f131246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:02.648601 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5754768b-e4f5-4a95-a0be-497d9f131246 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:02.648886 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5754768b-e4f5-4a95-a0be-497d9f131246 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:02.649164 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5754768b-e4f5-4a95-a0be-497d9f131246 admin admin] PATH_INFO: `/v3/projects/9d4a920f844d410eb6e42ae302d134c2` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:02.650466 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5754768b-e4f5-4a95-a0be-497d9f131246 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d4a920f844d410eb6e42ae302d134c2)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:02.652790 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5754768b-e4f5-4a95-a0be-497d9f131246 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:02.707496 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-5754768b-e4f5-4a95-a0be-497d9f131246 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d4a920f844d410eb6e42ae302d134c2', 'request_id': 'req-5754768b-e4f5-4a95-a0be-497d9f131246'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:46:02.715401 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-5754768b-e4f5-4a95-a0be-497d9f131246 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d4a920f844d410eb6e42ae302d134c2', 'request_id': 'req-5754768b-e4f5-4a95-a0be-497d9f131246'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:46:02.723784 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-5754768b-e4f5-4a95-a0be-497d9f131246 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d4a920f844d410eb6e42ae302d134c2', 'request_id': 'req-5754768b-e4f5-4a95-a0be-497d9f131246'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:46:02.735545 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-5754768b-e4f5-4a95-a0be-497d9f131246 admin admin] The token cache is being invalidate because project 9d4a920f844d410eb6e42ae302d134c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:46:02.735939 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-5754768b-e4f5-4a95-a0be-497d9f131246 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:46:02.741182 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-5754768b-e4f5-4a95-a0be-497d9f131246 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:46:02.743754 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1102/2183] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jun 16 17:46:02 2022] DELETE /identity/v3/projects/9d4a920f844d410eb6e42ae302d134c2 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:46:02.746980 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c31c602-c161-4e34-a3b0-293de688131d service neutron] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:02.759149 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c31c602-c161-4e34-a3b0-293de688131d service neutron] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:02.824121 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b43543-ad29-437e-8c94-12b81c054741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:02.825134 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b43543-ad29-437e-8c94-12b81c054741 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:02.825505 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b43543-ad29-437e-8c94-12b81c054741 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:02.826446 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b43543-ad29-437e-8c94-12b81c054741 admin admin] PATH_INFO: `/v3/users/32b1ece24aea460d8bb5c30e927ac5e5` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:02.827963 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b43543-ad29-437e-8c94-12b81c054741 admin admin] RBAC: Authorizing `identity:delete_user(user_id=32b1ece24aea460d8bb5c30e927ac5e5)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:02.830091 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b43543-ad29-437e-8c94-12b81c054741 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:02.876944 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5754768b-e4f5-4a95-a0be-497d9f131246 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:02.885096 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-5754768b-e4f5-4a95-a0be-497d9f131246 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:46:02.891568 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-47b43543-ad29-437e-8c94-12b81c054741 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32b1ece24aea460d8bb5c30e927ac5e5'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:46:02.892624 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-47b43543-ad29-437e-8c94-12b81c054741 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32b1ece24aea460d8bb5c30e927ac5e5'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:46:02.896904 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5754768b-e4f5-4a95-a0be-497d9f131246 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:02.901777 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-47b43543-ad29-437e-8c94-12b81c054741 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32b1ece24aea460d8bb5c30e927ac5e5'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:46:02.903048 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-47b43543-ad29-437e-8c94-12b81c054741 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32b1ece24aea460d8bb5c30e927ac5e5'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:46:02.903348 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2f8b35-fd14-4d80-99e0-52814766675d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:02.904934 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2f8b35-fd14-4d80-99e0-52814766675d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:02.905263 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2f8b35-fd14-4d80-99e0-52814766675d admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:02.905541 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2f8b35-fd14-4d80-99e0-52814766675d admin admin] PATH_INFO: `/v3/projects/e82c0414b4b64fed8fdcaacbbd84fa31` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:02.906805 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2f8b35-fd14-4d80-99e0-52814766675d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e82c0414b4b64fed8fdcaacbbd84fa31)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:02.909010 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2f8b35-fd14-4d80-99e0-52814766675d admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:02.920739 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-47b43543-ad29-437e-8c94-12b81c054741 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32b1ece24aea460d8bb5c30e927ac5e5'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:46:02.934659 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1082/2184] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jun 16 17:46:02 2022] DELETE /identity/v3/users/32b1ece24aea460d8bb5c30e927ac5e5 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:46:02.965673 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:02.977002 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ec2f8b35-fd14-4d80-99e0-52814766675d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e82c0414b4b64fed8fdcaacbbd84fa31', 'request_id': 'req-ec2f8b35-fd14-4d80-99e0-52814766675d'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:46:02.987365 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ec2f8b35-fd14-4d80-99e0-52814766675d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e82c0414b4b64fed8fdcaacbbd84fa31', 'request_id': 'req-ec2f8b35-fd14-4d80-99e0-52814766675d'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:46:02.996737 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ec2f8b35-fd14-4d80-99e0-52814766675d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e82c0414b4b64fed8fdcaacbbd84fa31', 'request_id': 'req-ec2f8b35-fd14-4d80-99e0-52814766675d'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:46:03.009858 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ec2f8b35-fd14-4d80-99e0-52814766675d admin admin] The token cache is being invalidate because project e82c0414b4b64fed8fdcaacbbd84fa31 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:46:03.010179 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ec2f8b35-fd14-4d80-99e0-52814766675d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:46:03.010850 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ec2f8b35-fd14-4d80-99e0-52814766675d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:46:03.013624 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1103/2185] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jun 16 17:46:02 2022] DELETE /identity/v3/projects/e82c0414b4b64fed8fdcaacbbd84fa31 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:46:03.018826 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1c2cf0-d6b3-416c-bf86-1a4425300805 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:03.019114 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1c2cf0-d6b3-416c-bf86-1a4425300805 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:03.019392 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1c2cf0-d6b3-416c-bf86-1a4425300805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:03.038007 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-8a1c2cf0-d6b3-416c-bf86-1a4425300805 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:46:03.039212 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-8a1c2cf0-d6b3-416c-bf86-1a4425300805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:46:03.105841 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1083/2186] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:46:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:46:03.114192 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b93635f-4f50-4e91-b081-6a81aa07474f None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:03.114483 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b93635f-4f50-4e91-b081-6a81aa07474f None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:03.114797 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b93635f-4f50-4e91-b081-6a81aa07474f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:03.134590 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-4b93635f-4f50-4e91-b081-6a81aa07474f None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:46:03.135795 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-4b93635f-4f50-4e91-b081-6a81aa07474f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:46:03.144535 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1104/2187] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:46:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:46:03.148979 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1c2cf0-d6b3-416c-bf86-1a4425300805 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:03.159752 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1c2cf0-d6b3-416c-bf86-1a4425300805 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:03.164519 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5abb9a-a4e5-48c2-894a-801e888a763c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:03.165422 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5abb9a-a4e5-48c2-894a-801e888a763c admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:03.165635 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5abb9a-a4e5-48c2-894a-801e888a763c admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:03.165911 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5abb9a-a4e5-48c2-894a-801e888a763c admin admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:03.166726 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5abb9a-a4e5-48c2-894a-801e888a763c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:46:03.167038 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5abb9a-a4e5-48c2-894a-801e888a763c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:03.168735 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5abb9a-a4e5-48c2-894a-801e888a763c admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:03.184242 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1084/2188] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jun 16 17:46:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:46:03.186044 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b93635f-4f50-4e91-b081-6a81aa07474f None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:03.195989 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b93635f-4f50-4e91-b081-6a81aa07474f None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:03.201452 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b3641d-67e3-414c-9bff-d6ac9d1d85b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:03.202312 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b3641d-67e3-414c-9bff-d6ac9d1d85b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:03.202632 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b3641d-67e3-414c-9bff-d6ac9d1d85b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:03.202917 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b3641d-67e3-414c-9bff-d6ac9d1d85b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:03.203619 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b3641d-67e3-414c-9bff-d6ac9d1d85b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:03.205554 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b3641d-67e3-414c-9bff-d6ac9d1d85b6 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:03.220271 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5abb9a-a4e5-48c2-894a-801e888a763c admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:03.222229 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-8a5abb9a-a4e5-48c2-894a-801e888a763c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:46:03.231624 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:03.239152 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5abb9a-a4e5-48c2-894a-801e888a763c admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:03.242755 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1105/2189] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jun 16 17:46:03 2022] POST /identity/v3/projects => generated 373 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:46:03.243635 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b3641d-67e3-414c-9bff-d6ac9d1d85b6 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:03.311826 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d5b7a5-ce93-4ba2-9274-00f379519a14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:03.312690 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b3641d-67e3-414c-9bff-d6ac9d1d85b6 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:03.312801 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d5b7a5-ce93-4ba2-9274-00f379519a14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:03.313002 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d5b7a5-ce93-4ba2-9274-00f379519a14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:03.313281 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d5b7a5-ce93-4ba2-9274-00f379519a14 admin admin] PATH_INFO: `/v3/projects/b77a6911f549462d8e458da87438003c` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:03.314462 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d5b7a5-ce93-4ba2-9274-00f379519a14 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b77a6911f549462d8e458da87438003c)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:03.316698 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d5b7a5-ce93-4ba2-9274-00f379519a14 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:03.317462 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eff56e4-db51-4be4-ba14-38ed40429ecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:03.318129 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eff56e4-db51-4be4-ba14-38ed40429ecd admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:03.318409 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eff56e4-db51-4be4-ba14-38ed40429ecd admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:03.318683 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eff56e4-db51-4be4-ba14-38ed40429ecd admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:03.319375 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eff56e4-db51-4be4-ba14-38ed40429ecd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:03.321316 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eff56e4-db51-4be4-ba14-38ed40429ecd admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:03.348001 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1106/2190] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jun 16 17:46:03 2022] POST /identity/v3/users => generated 287 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:46:03.349100 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eff56e4-db51-4be4-ba14-38ed40429ecd admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:03.359458 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eff56e4-db51-4be4-ba14-38ed40429ecd admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:03.368303 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-42d5b7a5-ce93-4ba2-9274-00f379519a14 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b77a6911f549462d8e458da87438003c', 'request_id': 'req-42d5b7a5-ce93-4ba2-9274-00f379519a14'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:46:03.377232 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-42d5b7a5-ce93-4ba2-9274-00f379519a14 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b77a6911f549462d8e458da87438003c', 'request_id': 'req-42d5b7a5-ce93-4ba2-9274-00f379519a14'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:46:03.383651 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-42d5b7a5-ce93-4ba2-9274-00f379519a14 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b77a6911f549462d8e458da87438003c', 'request_id': 'req-42d5b7a5-ce93-4ba2-9274-00f379519a14'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:46:03.398402 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-42d5b7a5-ce93-4ba2-9274-00f379519a14 admin admin] The token cache is being invalidate because project b77a6911f549462d8e458da87438003c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80357) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:46:03.398872 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-42d5b7a5-ce93-4ba2-9274-00f379519a14 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:46:03.399874 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-42d5b7a5-ce93-4ba2-9274-00f379519a14 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:46:03.402379 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1085/2191] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jun 16 17:46:03 2022] DELETE /identity/v3/projects/b77a6911f549462d8e458da87438003c => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:46:03.408457 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8ff4f0-b2e9-4adc-bdf9-5dc4cf82d2d3 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:03.408756 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8ff4f0-b2e9-4adc-bdf9-5dc4cf82d2d3 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:03.409038 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8ff4f0-b2e9-4adc-bdf9-5dc4cf82d2d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:03.421282 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e96c38-19fb-48bd-a3d6-833f6a806907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:03.421984 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e96c38-19fb-48bd-a3d6-833f6a806907 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:03.422283 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e96c38-19fb-48bd-a3d6-833f6a806907 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:03.422540 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e96c38-19fb-48bd-a3d6-833f6a806907 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:03.423382 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e96c38-19fb-48bd-a3d6-833f6a806907 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:03.425197 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e96c38-19fb-48bd-a3d6-833f6a806907 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:03.428446 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-1a8ff4f0-b2e9-4adc-bdf9-5dc4cf82d2d3 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:46:03.429842 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-1a8ff4f0-b2e9-4adc-bdf9-5dc4cf82d2d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:46:03.436944 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1107/2192] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:46:03 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 17:46:03.440763 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e96c38-19fb-48bd-a3d6-833f6a806907 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:03.442369 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-b9e96c38-19fb-48bd-a3d6-833f6a806907 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:46:03.442507 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1086/2193] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:46:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:46:03.451122 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170c6d59-807a-45ff-9ecd-03f896b62964 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:03.451857 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170c6d59-807a-45ff-9ecd-03f896b62964 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:03.451857 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170c6d59-807a-45ff-9ecd-03f896b62964 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:03.455853 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e96c38-19fb-48bd-a3d6-833f6a806907 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:03.461449 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc5ab60-929a-4f23-a111-b23a0e69df9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:03.462192 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc5ab60-929a-4f23-a111-b23a0e69df9f admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:03.462477 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc5ab60-929a-4f23-a111-b23a0e69df9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:03.462756 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc5ab60-929a-4f23-a111-b23a0e69df9f admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:03.463475 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc5ab60-929a-4f23-a111-b23a0e69df9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:03.465123 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc5ab60-929a-4f23-a111-b23a0e69df9f admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:03.469511 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-170c6d59-807a-45ff-9ecd-03f896b62964 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:46:03.471012 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-170c6d59-807a-45ff-9ecd-03f896b62964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:46:03.475633 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1108/2194] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:46:03 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 17:46:03.478765 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1087/2195] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:46:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:46:03.479733 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc5ab60-929a-4f23-a111-b23a0e69df9f admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:03.482709 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170c6d59-807a-45ff-9ecd-03f896b62964 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:03.489744 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc5ab60-929a-4f23-a111-b23a0e69df9f admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:03.492082 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170c6d59-807a-45ff-9ecd-03f896b62964 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:03.494505 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70821be7-584d-40a4-ab43-45588072fce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:03.495361 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70821be7-584d-40a4-ab43-45588072fce9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:03.495641 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70821be7-584d-40a4-ab43-45588072fce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:03.495915 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70821be7-584d-40a4-ab43-45588072fce9 admin admin] PATH_INFO: `/v3/projects/bb35ec3efb8c4a35b70191ebacce35a9/users/7e7032bdea794e57a7d6d1ad1cd2439e/roles/0be2075946f54c8485f1c275f746478a` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:03.497159 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221ce8ed-7f06-4a8e-9340-5ab966ddbbf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:03.498478 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221ce8ed-7f06-4a8e-9340-5ab966ddbbf8 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:03.498911 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221ce8ed-7f06-4a8e-9340-5ab966ddbbf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:03.499263 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221ce8ed-7f06-4a8e-9340-5ab966ddbbf8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:03.500181 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221ce8ed-7f06-4a8e-9340-5ab966ddbbf8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:46:03.500540 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221ce8ed-7f06-4a8e-9340-5ab966ddbbf8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:03.502143 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221ce8ed-7f06-4a8e-9340-5ab966ddbbf8 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:03.508819 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:03.514774 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70821be7-584d-40a4-ab43-45588072fce9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb35ec3efb8c4a35b70191ebacce35a9, user_id=7e7032bdea794e57a7d6d1ad1cd2439e, role_id=0be2075946f54c8485f1c275f746478a)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:03.515405 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1088/2196] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jun 16 17:46:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:46:03.518898 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70821be7-584d-40a4-ab43-45588072fce9 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:03.519539 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221ce8ed-7f06-4a8e-9340-5ab966ddbbf8 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:03.529927 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221ce8ed-7f06-4a8e-9340-5ab966ddbbf8 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:03.535113 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1109/2197] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:46:03 2022] PUT /identity/v3/projects/bb35ec3efb8c4a35b70191ebacce35a9/users/7e7032bdea794e57a7d6d1ad1cd2439e/roles/0be2075946f54c8485f1c275f746478a => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:46:03.546445 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d59e758-0e47-41f5-b5e3-9c7c80cd1166 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:03.546759 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d59e758-0e47-41f5-b5e3-9c7c80cd1166 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:03.547017 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d59e758-0e47-41f5-b5e3-9c7c80cd1166 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:03.577321 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-1d59e758-0e47-41f5-b5e3-9c7c80cd1166 None None] MFA Rules not processed for user `7e7032bdea794e57a7d6d1ad1cd2439e`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:46:03.578545 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-1d59e758-0e47-41f5-b5e3-9c7c80cd1166 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:46:03.591800 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f62f766-294d-4b9b-b881-4715520b39b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:03.592410 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f62f766-294d-4b9b-b881-4715520b39b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:03.592823 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f62f766-294d-4b9b-b881-4715520b39b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:03.592939 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f62f766-294d-4b9b-b881-4715520b39b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:03.593630 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f62f766-294d-4b9b-b881-4715520b39b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:03.595535 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f62f766-294d-4b9b-b881-4715520b39b9 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:03.607774 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:03.631067 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1089/2198] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jun 16 17:46:03 2022] POST /identity/v3/projects => generated 373 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:46:03.635653 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f62f766-294d-4b9b-b881-4715520b39b9 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:03.659968 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1110/2199] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:46:03 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:46:03.671840 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d59e758-0e47-41f5-b5e3-9c7c80cd1166 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:03.673483 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-1d59e758-0e47-41f5-b5e3-9c7c80cd1166 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:46:03.715869 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f62f766-294d-4b9b-b881-4715520b39b9 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:03.716757 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e12308-8207-4cbd-8e1a-1e30c425f7ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:03.717849 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e12308-8207-4cbd-8e1a-1e30c425f7ed admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:03.718140 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e12308-8207-4cbd-8e1a-1e30c425f7ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:03.718412 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e12308-8207-4cbd-8e1a-1e30c425f7ed admin admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:03.719097 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e12308-8207-4cbd-8e1a-1e30c425f7ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:03.721135 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e12308-8207-4cbd-8e1a-1e30c425f7ed admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:03.736136 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d59e758-0e47-41f5-b5e3-9c7c80cd1166 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:03.741960 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1090/2200] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jun 16 17:46:03 2022] POST /identity/v3/users => generated 287 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:46:03.745842 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e12308-8207-4cbd-8e1a-1e30c425f7ed admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:03.756232 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e12308-8207-4cbd-8e1a-1e30c425f7ed admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:03.761315 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2788410-a30a-4d4a-8730-903a3a740e18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:03.761966 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2788410-a30a-4d4a-8730-903a3a740e18 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:03.762253 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2788410-a30a-4d4a-8730-903a3a740e18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:03.762528 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2788410-a30a-4d4a-8730-903a3a740e18 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:03.763779 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2788410-a30a-4d4a-8730-903a3a740e18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:03.765602 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2788410-a30a-4d4a-8730-903a3a740e18 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:03.776517 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1091/2201] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:46:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 17:46:03.781188 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2788410-a30a-4d4a-8730-903a3a740e18 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:03.790840 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138b1e6c-7b25-4461-9d72-fe8ae8ae9652 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24d5e2637b7043e08313909d5b06e329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:03.791265 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2788410-a30a-4d4a-8730-903a3a740e18 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:03.791487 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138b1e6c-7b25-4461-9d72-fe8ae8ae9652 service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:03.791787 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138b1e6c-7b25-4461-9d72-fe8ae8ae9652 service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:03.792049 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138b1e6c-7b25-4461-9d72-fe8ae8ae9652 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:03.796242 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ed459b-eae5-4dd5-bf42-82d1e7773e36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:03.796920 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ed459b-eae5-4dd5-bf42-82d1e7773e36 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:03.797208 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ed459b-eae5-4dd5-bf42-82d1e7773e36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:03.797486 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ed459b-eae5-4dd5-bf42-82d1e7773e36 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:03.798270 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ed459b-eae5-4dd5-bf42-82d1e7773e36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:03.799907 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ed459b-eae5-4dd5-bf42-82d1e7773e36 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:03.810659 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1092/2202] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:46:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 17:46:03.814795 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ed459b-eae5-4dd5-bf42-82d1e7773e36 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:03.824281 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ed459b-eae5-4dd5-bf42-82d1e7773e36 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:03.829795 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7339e35-cd07-4ed1-808f-ca18a4417400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:03.830657 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7339e35-cd07-4ed1-808f-ca18a4417400 admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:03.830936 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7339e35-cd07-4ed1-808f-ca18a4417400 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:03.831208 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7339e35-cd07-4ed1-808f-ca18a4417400 admin admin] PATH_INFO: `/v3/projects/468d1ba1f11f46448d9ef415e048f0f9/users/84e05e8f65a740fd96a913c7efcea611/roles/0be2075946f54c8485f1c275f746478a` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:03.841054 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:03.848575 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7339e35-cd07-4ed1-808f-ca18a4417400 admin admin] RBAC: Authorizing `identity:create_grant(project_id=468d1ba1f11f46448d9ef415e048f0f9, user_id=84e05e8f65a740fd96a913c7efcea611, role_id=0be2075946f54c8485f1c275f746478a)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:03.851250 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7339e35-cd07-4ed1-808f-ca18a4417400 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:03.853102 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138b1e6c-7b25-4461-9d72-fe8ae8ae9652 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:03.854646 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138b1e6c-7b25-4461-9d72-fe8ae8ae9652 service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:03.863906 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1093/2203] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:46:03 2022] PUT /identity/v3/projects/468d1ba1f11f46448d9ef415e048f0f9/users/84e05e8f65a740fd96a913c7efcea611/roles/0be2075946f54c8485f1c275f746478a => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:46:03.864855 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1111/2204] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:46:03 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:46:03.875444 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fab04f-57e5-44d4-9410-756b43916970 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:03.875873 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fab04f-57e5-44d4-9410-756b43916970 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:03.876152 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fab04f-57e5-44d4-9410-756b43916970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:03.905738 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-b6fab04f-57e5-44d4-9410-756b43916970 None None] MFA Rules not processed for user `84e05e8f65a740fd96a913c7efcea611`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:46:03.906950 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-b6fab04f-57e5-44d4-9410-756b43916970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:46:03.975647 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1094/2205] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:46:03 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:46:03.986562 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138b1e6c-7b25-4461-9d72-fe8ae8ae9652 service cinder] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:03.988154 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-138b1e6c-7b25-4461-9d72-fe8ae8ae9652 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:46:04.001050 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138b1e6c-7b25-4461-9d72-fe8ae8ae9652 service cinder] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:04.054822 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca08449-ec1d-4e73-b828-3378655bfd29 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24d5e2637b7043e08313909d5b06e329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:04.055609 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca08449-ec1d-4e73-b828-3378655bfd29 service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:04.055749 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca08449-ec1d-4e73-b828-3378655bfd29 service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:04.055998 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca08449-ec1d-4e73-b828-3378655bfd29 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:04.065000 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca08449-ec1d-4e73-b828-3378655bfd29 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:04.066428 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca08449-ec1d-4e73-b828-3378655bfd29 service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:04.077105 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1112/2206] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:46:03 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:46:06.271408 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e76950a-e8d5-41bf-acd6-39aac2a579cd None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:06.272133 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e76950a-e8d5-41bf-acd6-39aac2a579cd None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:06.272133 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e76950a-e8d5-41bf-acd6-39aac2a579cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:06.291180 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-6e76950a-e8d5-41bf-acd6-39aac2a579cd None None] MFA Rules not processed for user `7e7032bdea794e57a7d6d1ad1cd2439e`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:46:06.292468 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-6e76950a-e8d5-41bf-acd6-39aac2a579cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:46:06.315016 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:06.358202 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1095/2207] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:46:06 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:46:11.361786 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca08449-ec1d-4e73-b828-3378655bfd29 service cinder] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:11.429096 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca08449-ec1d-4e73-b828-3378655bfd29 service cinder] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:11.433972 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c418409-fbfa-4899-9661-cb07dfec3c58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:11.434845 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c418409-fbfa-4899-9661-cb07dfec3c58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:11.435126 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c418409-fbfa-4899-9661-cb07dfec3c58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:11.435403 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c418409-fbfa-4899-9661-cb07dfec3c58 admin admin] PATH_INFO: `/v3/users/84e05e8f65a740fd96a913c7efcea611` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:11.436667 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c418409-fbfa-4899-9661-cb07dfec3c58 admin admin] RBAC: Authorizing `identity:delete_user(user_id=84e05e8f65a740fd96a913c7efcea611)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:11.438621 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c418409-fbfa-4899-9661-cb07dfec3c58 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:11.483762 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-9c418409-fbfa-4899-9661-cb07dfec3c58 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84e05e8f65a740fd96a913c7efcea611'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:46:11.495386 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-9c418409-fbfa-4899-9661-cb07dfec3c58 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84e05e8f65a740fd96a913c7efcea611'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:46:11.496057 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-9c418409-fbfa-4899-9661-cb07dfec3c58 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84e05e8f65a740fd96a913c7efcea611'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:46:11.503054 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-9c418409-fbfa-4899-9661-cb07dfec3c58 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84e05e8f65a740fd96a913c7efcea611'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:46:11.510876 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-9c418409-fbfa-4899-9661-cb07dfec3c58 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84e05e8f65a740fd96a913c7efcea611'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:46:11.518495 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1113/2208] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jun 16 17:46:11 2022] DELETE /identity/v3/users/84e05e8f65a740fd96a913c7efcea611 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:46:11.526854 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9c2716-0308-4f39-9ce3-adc35ba268d0 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:11.527305 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9c2716-0308-4f39-9ce3-adc35ba268d0 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:11.527417 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9c2716-0308-4f39-9ce3-adc35ba268d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:11.545236 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-cc9c2716-0308-4f39-9ce3-adc35ba268d0 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:46:11.546428 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-cc9c2716-0308-4f39-9ce3-adc35ba268d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:46:11.604012 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1096/2209] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:46:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:46:11.613659 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c418409-fbfa-4899-9661-cb07dfec3c58 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:11.615056 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-9c418409-fbfa-4899-9661-cb07dfec3c58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:46:11.629038 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:11.677359 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c418409-fbfa-4899-9661-cb07dfec3c58 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:11.681041 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c4d7b8-23ba-436a-aa03-f193716fe978 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:11.681681 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c4d7b8-23ba-436a-aa03-f193716fe978 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:11.681957 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c4d7b8-23ba-436a-aa03-f193716fe978 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:11.682228 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c4d7b8-23ba-436a-aa03-f193716fe978 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:11.691810 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c4d7b8-23ba-436a-aa03-f193716fe978 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:11.693259 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c4d7b8-23ba-436a-aa03-f193716fe978 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:11.703204 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1114/2210] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:46:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:46:12.049216 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9c2716-0308-4f39-9ce3-adc35ba268d0 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:12.050661 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-cc9c2716-0308-4f39-9ce3-adc35ba268d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:46:12.063285 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9c2716-0308-4f39-9ce3-adc35ba268d0 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:12.067869 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b5cddb-f578-4216-ae56-52e76bfd3ece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:12.068775 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b5cddb-f578-4216-ae56-52e76bfd3ece admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:12.069045 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b5cddb-f578-4216-ae56-52e76bfd3ece admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:12.069327 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b5cddb-f578-4216-ae56-52e76bfd3ece admin admin] PATH_INFO: `/v3/projects/468d1ba1f11f46448d9ef415e048f0f9` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:12.070520 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b5cddb-f578-4216-ae56-52e76bfd3ece admin admin] RBAC: Authorizing `identity:delete_project(project_id=468d1ba1f11f46448d9ef415e048f0f9)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:12.072509 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b5cddb-f578-4216-ae56-52e76bfd3ece admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:12.121133 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-d1b5cddb-f578-4216-ae56-52e76bfd3ece admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '468d1ba1f11f46448d9ef415e048f0f9', 'request_id': 'req-d1b5cddb-f578-4216-ae56-52e76bfd3ece'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:46:12.130287 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-d1b5cddb-f578-4216-ae56-52e76bfd3ece admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '468d1ba1f11f46448d9ef415e048f0f9', 'request_id': 'req-d1b5cddb-f578-4216-ae56-52e76bfd3ece'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:46:12.136679 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-d1b5cddb-f578-4216-ae56-52e76bfd3ece admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '468d1ba1f11f46448d9ef415e048f0f9', 'request_id': 'req-d1b5cddb-f578-4216-ae56-52e76bfd3ece'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:46:12.148326 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-d1b5cddb-f578-4216-ae56-52e76bfd3ece admin admin] The token cache is being invalidate because project 468d1ba1f11f46448d9ef415e048f0f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80357) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:46:12.148642 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-d1b5cddb-f578-4216-ae56-52e76bfd3ece admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:46:12.149296 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-d1b5cddb-f578-4216-ae56-52e76bfd3ece admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:46:12.150832 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1097/2211] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jun 16 17:46:12 2022] DELETE /identity/v3/projects/468d1ba1f11f46448d9ef415e048f0f9 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:46:12.157186 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bc9fae-79c0-45ab-acd0-0cf7cbe04bd8 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:12.157550 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bc9fae-79c0-45ab-acd0-0cf7cbe04bd8 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:12.157902 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bc9fae-79c0-45ab-acd0-0cf7cbe04bd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:12.186855 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-e7bc9fae-79c0-45ab-acd0-0cf7cbe04bd8 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:46:12.188040 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-e7bc9fae-79c0-45ab-acd0-0cf7cbe04bd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:46:12.250739 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1115/2212] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:46:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:46:12.258904 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51102adf-ad6e-403e-b316-47a7219109ff None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:12.259275 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51102adf-ad6e-403e-b316-47a7219109ff None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:12.259642 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51102adf-ad6e-403e-b316-47a7219109ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:12.278155 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-51102adf-ad6e-403e-b316-47a7219109ff None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:46:12.279391 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-51102adf-ad6e-403e-b316-47a7219109ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:46:12.287317 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1098/2213] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:46:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:46:12.291569 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bc9fae-79c0-45ab-acd0-0cf7cbe04bd8 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:12.302036 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bc9fae-79c0-45ab-acd0-0cf7cbe04bd8 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:12.306629 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cf1b5a-f362-4cb6-8f63-56548ebd9286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:12.307280 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cf1b5a-f362-4cb6-8f63-56548ebd9286 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:12.307565 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cf1b5a-f362-4cb6-8f63-56548ebd9286 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:12.307843 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cf1b5a-f362-4cb6-8f63-56548ebd9286 admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:12.308594 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cf1b5a-f362-4cb6-8f63-56548ebd9286 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:46:12.309197 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cf1b5a-f362-4cb6-8f63-56548ebd9286 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:12.311523 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cf1b5a-f362-4cb6-8f63-56548ebd9286 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:12.322963 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1116/2214] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jun 16 17:46:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:46:12.327443 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51102adf-ad6e-403e-b316-47a7219109ff None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:12.337941 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51102adf-ad6e-403e-b316-47a7219109ff None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:12.344362 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59aad12e-1bfa-4b8e-921d-f3e788eb3ae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:12.345210 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aad12e-1bfa-4b8e-921d-f3e788eb3ae7 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:12.345569 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aad12e-1bfa-4b8e-921d-f3e788eb3ae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:12.346040 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aad12e-1bfa-4b8e-921d-f3e788eb3ae7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:12.346816 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59aad12e-1bfa-4b8e-921d-f3e788eb3ae7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:12.348950 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59aad12e-1bfa-4b8e-921d-f3e788eb3ae7 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:12.381737 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1099/2215] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jun 16 17:46:12 2022] POST /identity/v3/projects => generated 373 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:46:12.386416 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cf1b5a-f362-4cb6-8f63-56548ebd9286 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:12.411389 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cf1b5a-f362-4cb6-8f63-56548ebd9286 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:12.487256 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6be80d-7c7b-4d49-b62d-c8bc05a90586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:12.487939 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6be80d-7c7b-4d49-b62d-c8bc05a90586 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:12.488227 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6be80d-7c7b-4d49-b62d-c8bc05a90586 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:12.488548 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6be80d-7c7b-4d49-b62d-c8bc05a90586 admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:12.489246 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6be80d-7c7b-4d49-b62d-c8bc05a90586 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:12.491155 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6be80d-7c7b-4d49-b62d-c8bc05a90586 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:12.512683 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1117/2216] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jun 16 17:46:12 2022] POST /identity/v3/users => generated 287 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:46:12.517672 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59aad12e-1bfa-4b8e-921d-f3e788eb3ae7 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:12.529264 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59aad12e-1bfa-4b8e-921d-f3e788eb3ae7 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:12.534528 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ca442c-af83-4fca-a8c5-4cc1c67cf3c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:12.535198 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ca442c-af83-4fca-a8c5-4cc1c67cf3c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:12.535480 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ca442c-af83-4fca-a8c5-4cc1c67cf3c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:12.535784 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ca442c-af83-4fca-a8c5-4cc1c67cf3c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:12.536549 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ca442c-af83-4fca-a8c5-4cc1c67cf3c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:12.538180 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ca442c-af83-4fca-a8c5-4cc1c67cf3c9 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:12.549895 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1100/2217] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:46:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:46:12.553989 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6be80d-7c7b-4d49-b62d-c8bc05a90586 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:12.565636 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6be80d-7c7b-4d49-b62d-c8bc05a90586 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:12.570650 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9e136a-b8a3-4d5f-bad8-ac2a8afcd5b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:12.571319 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9e136a-b8a3-4d5f-bad8-ac2a8afcd5b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:12.571603 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9e136a-b8a3-4d5f-bad8-ac2a8afcd5b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:12.571879 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9e136a-b8a3-4d5f-bad8-ac2a8afcd5b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:12.572628 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9e136a-b8a3-4d5f-bad8-ac2a8afcd5b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:12.574251 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9e136a-b8a3-4d5f-bad8-ac2a8afcd5b7 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:12.585972 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1118/2218] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:46:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:46:12.590216 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ca442c-af83-4fca-a8c5-4cc1c67cf3c9 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:12.601290 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ca442c-af83-4fca-a8c5-4cc1c67cf3c9 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:12.606013 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabf0b72-76ec-4b92-92f2-698963024ef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:12.607770 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabf0b72-76ec-4b92-92f2-698963024ef3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:12.608064 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabf0b72-76ec-4b92-92f2-698963024ef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:12.608350 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabf0b72-76ec-4b92-92f2-698963024ef3 admin admin] PATH_INFO: `/v3/projects/c1a3f0c9240b45ac8ac2e9714d62ab8b/users/892a5585213c4b4fadba01bc5fbba767/roles/0be2075946f54c8485f1c275f746478a` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:12.619657 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:12.625033 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabf0b72-76ec-4b92-92f2-698963024ef3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1a3f0c9240b45ac8ac2e9714d62ab8b, user_id=892a5585213c4b4fadba01bc5fbba767, role_id=0be2075946f54c8485f1c275f746478a)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:12.627806 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabf0b72-76ec-4b92-92f2-698963024ef3 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:12.642706 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1101/2219] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:46:12 2022] PUT /identity/v3/projects/c1a3f0c9240b45ac8ac2e9714d62ab8b/users/892a5585213c4b4fadba01bc5fbba767/roles/0be2075946f54c8485f1c275f746478a => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:46:12.654060 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d8ff38-959d-487b-aa0e-ebfa388d1a68 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:12.654358 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d8ff38-959d-487b-aa0e-ebfa388d1a68 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:12.654637 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d8ff38-959d-487b-aa0e-ebfa388d1a68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:12.674579 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:12.682860 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-48d8ff38-959d-487b-aa0e-ebfa388d1a68 None None] MFA Rules not processed for user `892a5585213c4b4fadba01bc5fbba767`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:46:12.684514 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-48d8ff38-959d-487b-aa0e-ebfa388d1a68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:46:12.749604 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1119/2220] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:46:12 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:46:12.761892 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabf0b72-76ec-4b92-92f2-698963024ef3 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:12.764857 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-dabf0b72-76ec-4b92-92f2-698963024ef3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:46:12.825876 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabf0b72-76ec-4b92-92f2-698963024ef3 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:12.829907 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f318a5f4-a6b5-44ba-a230-9492326ed6a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24d5e2637b7043e08313909d5b06e329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:12.830552 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f318a5f4-a6b5-44ba-a230-9492326ed6a7 service cinder] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:12.830836 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f318a5f4-a6b5-44ba-a230-9492326ed6a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:12.831115 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f318a5f4-a6b5-44ba-a230-9492326ed6a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:12.839417 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f318a5f4-a6b5-44ba-a230-9492326ed6a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:12.841986 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f318a5f4-a6b5-44ba-a230-9492326ed6a7 service cinder] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:12.851130 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1102/2221] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:46:12 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:46:19.984080 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d8ff38-959d-487b-aa0e-ebfa388d1a68 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:19.985593 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-48d8ff38-959d-487b-aa0e-ebfa388d1a68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:46:20.069948 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d8ff38-959d-487b-aa0e-ebfa388d1a68 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:20.080513 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2054eb77-2ebe-452b-a141-e9f8ba6767aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:20.081538 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2054eb77-2ebe-452b-a141-e9f8ba6767aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:20.081923 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2054eb77-2ebe-452b-a141-e9f8ba6767aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:20.082273 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2054eb77-2ebe-452b-a141-e9f8ba6767aa admin admin] PATH_INFO: `/v3/users/b6e833a13903498eb5a49bc0996a4da0` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:20.087730 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2054eb77-2ebe-452b-a141-e9f8ba6767aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6e833a13903498eb5a49bc0996a4da0)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:20.087730 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2054eb77-2ebe-452b-a141-e9f8ba6767aa admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:20.165460 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-2054eb77-2ebe-452b-a141-e9f8ba6767aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6e833a13903498eb5a49bc0996a4da0'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:46:20.180634 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-2054eb77-2ebe-452b-a141-e9f8ba6767aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6e833a13903498eb5a49bc0996a4da0'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:46:20.181383 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-2054eb77-2ebe-452b-a141-e9f8ba6767aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6e833a13903498eb5a49bc0996a4da0'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:46:20.193414 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-2054eb77-2ebe-452b-a141-e9f8ba6767aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6e833a13903498eb5a49bc0996a4da0'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:46:20.201497 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-2054eb77-2ebe-452b-a141-e9f8ba6767aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6e833a13903498eb5a49bc0996a4da0'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:46:20.210397 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1120/2222] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jun 16 17:46:19 2022] DELETE /identity/v3/users/b6e833a13903498eb5a49bc0996a4da0 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:46:20.215560 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc002a0-09b8-4747-941c-96291a98ba20 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:20.216049 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc002a0-09b8-4747-941c-96291a98ba20 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:20.216144 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc002a0-09b8-4747-941c-96291a98ba20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:20.237471 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-8dc002a0-09b8-4747-941c-96291a98ba20 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:46:20.238717 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-8dc002a0-09b8-4747-941c-96291a98ba20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:46:20.313761 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1103/2223] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:46:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:46:20.323879 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2054eb77-2ebe-452b-a141-e9f8ba6767aa admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:20.326043 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-2054eb77-2ebe-452b-a141-e9f8ba6767aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:46:20.371906 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:20.432963 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2054eb77-2ebe-452b-a141-e9f8ba6767aa admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:20.437742 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f2baad-c00b-4c9f-8480-52baafd1ec43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:20.438533 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f2baad-c00b-4c9f-8480-52baafd1ec43 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:20.438862 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f2baad-c00b-4c9f-8480-52baafd1ec43 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:20.439229 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f2baad-c00b-4c9f-8480-52baafd1ec43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:20.451835 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f2baad-c00b-4c9f-8480-52baafd1ec43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:20.454141 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f2baad-c00b-4c9f-8480-52baafd1ec43 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:20.467062 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1121/2224] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:46:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:46:20.881131 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc002a0-09b8-4747-941c-96291a98ba20 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:20.882908 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-8dc002a0-09b8-4747-941c-96291a98ba20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:46:20.901178 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc002a0-09b8-4747-941c-96291a98ba20 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:20.907380 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d30d13-0c4e-4fe2-aeb8-da195bbd91dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:20.908240 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d30d13-0c4e-4fe2-aeb8-da195bbd91dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:20.908557 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d30d13-0c4e-4fe2-aeb8-da195bbd91dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:20.908862 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d30d13-0c4e-4fe2-aeb8-da195bbd91dd admin admin] PATH_INFO: `/v3/projects/a697b09fbb1845c0b61a090ea1083912` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:20.910107 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d30d13-0c4e-4fe2-aeb8-da195bbd91dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=a697b09fbb1845c0b61a090ea1083912)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:20.912491 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d30d13-0c4e-4fe2-aeb8-da195bbd91dd admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:20.966420 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-a0d30d13-0c4e-4fe2-aeb8-da195bbd91dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a697b09fbb1845c0b61a090ea1083912', 'request_id': 'req-a0d30d13-0c4e-4fe2-aeb8-da195bbd91dd'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:46:20.976588 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-a0d30d13-0c4e-4fe2-aeb8-da195bbd91dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a697b09fbb1845c0b61a090ea1083912', 'request_id': 'req-a0d30d13-0c4e-4fe2-aeb8-da195bbd91dd'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:46:20.982607 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-a0d30d13-0c4e-4fe2-aeb8-da195bbd91dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a697b09fbb1845c0b61a090ea1083912', 'request_id': 'req-a0d30d13-0c4e-4fe2-aeb8-da195bbd91dd'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:46:20.999202 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-a0d30d13-0c4e-4fe2-aeb8-da195bbd91dd admin admin] The token cache is being invalidate because project a697b09fbb1845c0b61a090ea1083912 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80357) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:46:20.999704 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-a0d30d13-0c4e-4fe2-aeb8-da195bbd91dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:46:21.000511 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-a0d30d13-0c4e-4fe2-aeb8-da195bbd91dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:46:21.002705 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1104/2225] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jun 16 17:46:20 2022] DELETE /identity/v3/projects/a697b09fbb1845c0b61a090ea1083912 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:46:21.007326 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7718ad-067b-41ac-b989-737d763e2dff None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:21.007827 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7718ad-067b-41ac-b989-737d763e2dff None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:21.007946 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7718ad-067b-41ac-b989-737d763e2dff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:21.028268 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-ce7718ad-067b-41ac-b989-737d763e2dff None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:46:21.029714 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-ce7718ad-067b-41ac-b989-737d763e2dff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:46:21.098201 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1122/2226] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:46:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:46:21.106094 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a7c531-6be2-4d46-ba12-8c9983408625 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:21.106409 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a7c531-6be2-4d46-ba12-8c9983408625 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:21.106777 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a7c531-6be2-4d46-ba12-8c9983408625 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:21.126833 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-40a7c531-6be2-4d46-ba12-8c9983408625 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:46:21.128064 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-40a7c531-6be2-4d46-ba12-8c9983408625 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:46:21.136621 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1105/2227] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:46:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:46:21.140880 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7718ad-067b-41ac-b989-737d763e2dff None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:21.150497 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7718ad-067b-41ac-b989-737d763e2dff None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:21.155138 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9ce770-b344-42db-aee8-de475660efe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:21.155759 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9ce770-b344-42db-aee8-de475660efe7 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:21.156033 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9ce770-b344-42db-aee8-de475660efe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:21.156310 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9ce770-b344-42db-aee8-de475660efe7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:21.157057 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9ce770-b344-42db-aee8-de475660efe7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:46:21.157368 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9ce770-b344-42db-aee8-de475660efe7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:21.158970 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9ce770-b344-42db-aee8-de475660efe7 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:21.170311 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1123/2228] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jun 16 17:46:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:46:21.175008 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a7c531-6be2-4d46-ba12-8c9983408625 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:21.184275 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a7c531-6be2-4d46-ba12-8c9983408625 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:21.189650 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d204ea94-1df3-4680-9015-4b5294af76eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:21.190301 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d204ea94-1df3-4680-9015-4b5294af76eb admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:21.190592 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d204ea94-1df3-4680-9015-4b5294af76eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:21.190871 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d204ea94-1df3-4680-9015-4b5294af76eb admin admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:21.191552 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d204ea94-1df3-4680-9015-4b5294af76eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:21.193500 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d204ea94-1df3-4680-9015-4b5294af76eb admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:21.197561 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:21.226627 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1106/2229] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jun 16 17:46:21 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:46:21.227792 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9ce770-b344-42db-aee8-de475660efe7 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:21.238833 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9ce770-b344-42db-aee8-de475660efe7 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:21.303375 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef6c926-22d5-46d4-b47d-503c7e2712ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:21.304199 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef6c926-22d5-46d4-b47d-503c7e2712ad admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:21.304330 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef6c926-22d5-46d4-b47d-503c7e2712ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:21.304646 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef6c926-22d5-46d4-b47d-503c7e2712ad admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:21.305351 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef6c926-22d5-46d4-b47d-503c7e2712ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:21.307256 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef6c926-22d5-46d4-b47d-503c7e2712ad admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:21.336391 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1124/2230] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jun 16 17:46:21 2022] POST /identity/v3/users => generated 293 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:46:21.340530 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d204ea94-1df3-4680-9015-4b5294af76eb admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:21.352177 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d204ea94-1df3-4680-9015-4b5294af76eb admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:21.363105 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16f51dc-01ec-4f57-9b1f-c7a50bcd3da1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:21.363807 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16f51dc-01ec-4f57-9b1f-c7a50bcd3da1 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:21.364097 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16f51dc-01ec-4f57-9b1f-c7a50bcd3da1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:21.364404 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16f51dc-01ec-4f57-9b1f-c7a50bcd3da1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:21.365173 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16f51dc-01ec-4f57-9b1f-c7a50bcd3da1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:21.366860 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16f51dc-01ec-4f57-9b1f-c7a50bcd3da1 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:21.385469 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1107/2231] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:46:21 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:46:21.389417 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef6c926-22d5-46d4-b47d-503c7e2712ad admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:21.399577 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef6c926-22d5-46d4-b47d-503c7e2712ad admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:21.404254 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e101d4-970c-4d84-91c6-9fbb716694b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:21.404950 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e101d4-970c-4d84-91c6-9fbb716694b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:21.405805 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e101d4-970c-4d84-91c6-9fbb716694b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:21.406064 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e101d4-970c-4d84-91c6-9fbb716694b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:21.406789 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e101d4-970c-4d84-91c6-9fbb716694b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:21.408409 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e101d4-970c-4d84-91c6-9fbb716694b7 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:21.418229 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1125/2232] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:46:21 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:46:21.423415 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16f51dc-01ec-4f57-9b1f-c7a50bcd3da1 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:21.434388 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16f51dc-01ec-4f57-9b1f-c7a50bcd3da1 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:21.439314 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce45b65a-92c0-4f3a-8cc4-9242e480a333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:21.440257 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce45b65a-92c0-4f3a-8cc4-9242e480a333 admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:21.440572 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce45b65a-92c0-4f3a-8cc4-9242e480a333 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:21.440847 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce45b65a-92c0-4f3a-8cc4-9242e480a333 admin admin] PATH_INFO: `/v3/projects/ac4c34d742f64d17b81d6a6171e66e84/users/03be2d45633949a7b416fc5a84b8f000/roles/0be2075946f54c8485f1c275f746478a` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:21.455134 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce45b65a-92c0-4f3a-8cc4-9242e480a333 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac4c34d742f64d17b81d6a6171e66e84, user_id=03be2d45633949a7b416fc5a84b8f000, role_id=0be2075946f54c8485f1c275f746478a)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:21.457879 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce45b65a-92c0-4f3a-8cc4-9242e480a333 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:21.471159 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1108/2233] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:46:21 2022] PUT /identity/v3/projects/ac4c34d742f64d17b81d6a6171e66e84/users/03be2d45633949a7b416fc5a84b8f000/roles/0be2075946f54c8485f1c275f746478a => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:46:21.476251 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f404d7-a325-46bd-8b82-0fc1f91fb10b None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:21.476593 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f404d7-a325-46bd-8b82-0fc1f91fb10b None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:21.476882 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f404d7-a325-46bd-8b82-0fc1f91fb10b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:21.494865 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-e0f404d7-a325-46bd-8b82-0fc1f91fb10b None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:46:21.496067 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-e0f404d7-a325-46bd-8b82-0fc1f91fb10b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:46:21.520242 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:21.563734 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1126/2234] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:46:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:46:21.573764 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce45b65a-92c0-4f3a-8cc4-9242e480a333 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:21.575994 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-ce45b65a-92c0-4f3a-8cc4-9242e480a333 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:46:21.675220 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce45b65a-92c0-4f3a-8cc4-9242e480a333 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:21.678981 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cca1d4-ac8d-490c-a65c-cdd179a80ffc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:21.679833 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cca1d4-ac8d-490c-a65c-cdd179a80ffc service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:21.680237 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cca1d4-ac8d-490c-a65c-cdd179a80ffc service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:21.680653 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cca1d4-ac8d-490c-a65c-cdd179a80ffc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:21.692482 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cca1d4-ac8d-490c-a65c-cdd179a80ffc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:21.694183 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cca1d4-ac8d-490c-a65c-cdd179a80ffc service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:21.712686 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1109/2235] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:46:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:46:22.954887 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f404d7-a325-46bd-8b82-0fc1f91fb10b None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:22.956474 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-e0f404d7-a325-46bd-8b82-0fc1f91fb10b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:46:22.972153 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f404d7-a325-46bd-8b82-0fc1f91fb10b None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:22.978435 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72339e3-533c-49d0-bc5e-97757d8bb649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:22.979660 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72339e3-533c-49d0-bc5e-97757d8bb649 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:22.980258 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72339e3-533c-49d0-bc5e-97757d8bb649 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:22.980792 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72339e3-533c-49d0-bc5e-97757d8bb649 admin admin] PATH_INFO: `/v3/users/892a5585213c4b4fadba01bc5fbba767` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:22.982192 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72339e3-533c-49d0-bc5e-97757d8bb649 admin admin] RBAC: Authorizing `identity:delete_user(user_id=892a5585213c4b4fadba01bc5fbba767)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:22.984320 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72339e3-533c-49d0-bc5e-97757d8bb649 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:23.065855 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-c72339e3-533c-49d0-bc5e-97757d8bb649 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '892a5585213c4b4fadba01bc5fbba767'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:46:23.083909 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-c72339e3-533c-49d0-bc5e-97757d8bb649 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '892a5585213c4b4fadba01bc5fbba767'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:46:23.084844 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-c72339e3-533c-49d0-bc5e-97757d8bb649 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '892a5585213c4b4fadba01bc5fbba767'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:46:23.095072 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-c72339e3-533c-49d0-bc5e-97757d8bb649 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '892a5585213c4b4fadba01bc5fbba767'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:46:23.110270 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-c72339e3-533c-49d0-bc5e-97757d8bb649 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '892a5585213c4b4fadba01bc5fbba767'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:46:23.127355 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1127/2236] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jun 16 17:46:22 2022] DELETE /identity/v3/users/892a5585213c4b4fadba01bc5fbba767 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:46:23.132097 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e8c376-bc3f-4f78-a477-e4d775ff7402 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:23.132605 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e8c376-bc3f-4f78-a477-e4d775ff7402 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:23.132706 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e8c376-bc3f-4f78-a477-e4d775ff7402 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:23.184990 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-14e8c376-bc3f-4f78-a477-e4d775ff7402 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:46:23.190480 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-14e8c376-bc3f-4f78-a477-e4d775ff7402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:46:23.304819 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1110/2237] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:46:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:46:23.312166 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72339e3-533c-49d0-bc5e-97757d8bb649 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:23.313705 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-c72339e3-533c-49d0-bc5e-97757d8bb649 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:46:23.438002 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72339e3-533c-49d0-bc5e-97757d8bb649 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:23.445071 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e2efbc-a853-42db-ad66-acec5de4430b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:23.446529 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e2efbc-a853-42db-ad66-acec5de4430b service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:23.447120 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e2efbc-a853-42db-ad66-acec5de4430b service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:23.447576 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e2efbc-a853-42db-ad66-acec5de4430b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:23.470098 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e2efbc-a853-42db-ad66-acec5de4430b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:23.472930 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e2efbc-a853-42db-ad66-acec5de4430b service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:23.485448 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1128/2238] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:46:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:46:24.031647 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e8c376-bc3f-4f78-a477-e4d775ff7402 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:24.033779 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-14e8c376-bc3f-4f78-a477-e4d775ff7402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:46:24.053771 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e8c376-bc3f-4f78-a477-e4d775ff7402 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:24.061066 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38be7287-205e-421a-810f-1e45d205d232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:24.063258 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38be7287-205e-421a-810f-1e45d205d232 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:24.063651 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38be7287-205e-421a-810f-1e45d205d232 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:24.064036 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38be7287-205e-421a-810f-1e45d205d232 admin admin] PATH_INFO: `/v3/projects/c1a3f0c9240b45ac8ac2e9714d62ab8b` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:24.069434 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38be7287-205e-421a-810f-1e45d205d232 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1a3f0c9240b45ac8ac2e9714d62ab8b)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:24.072323 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38be7287-205e-421a-810f-1e45d205d232 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:24.156683 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-38be7287-205e-421a-810f-1e45d205d232 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1a3f0c9240b45ac8ac2e9714d62ab8b', 'request_id': 'req-38be7287-205e-421a-810f-1e45d205d232'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:46:24.158981 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:24.172517 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-38be7287-205e-421a-810f-1e45d205d232 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1a3f0c9240b45ac8ac2e9714d62ab8b', 'request_id': 'req-38be7287-205e-421a-810f-1e45d205d232'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:46:24.180710 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-38be7287-205e-421a-810f-1e45d205d232 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1a3f0c9240b45ac8ac2e9714d62ab8b', 'request_id': 'req-38be7287-205e-421a-810f-1e45d205d232'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:46:24.198722 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-38be7287-205e-421a-810f-1e45d205d232 admin admin] The token cache is being invalidate because project c1a3f0c9240b45ac8ac2e9714d62ab8b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80357) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:46:24.199255 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-38be7287-205e-421a-810f-1e45d205d232 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:46:24.200282 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-38be7287-205e-421a-810f-1e45d205d232 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:46:24.203798 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1111/2239] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jun 16 17:46:24 2022] DELETE /identity/v3/projects/c1a3f0c9240b45ac8ac2e9714d62ab8b => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:46:26.919111 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66023f2a-4287-479e-8885-a3e40ac5d6d0 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:26.919901 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66023f2a-4287-479e-8885-a3e40ac5d6d0 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:26.920493 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66023f2a-4287-479e-8885-a3e40ac5d6d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:26.964995 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-66023f2a-4287-479e-8885-a3e40ac5d6d0 None None] MFA Rules not processed for user `03be2d45633949a7b416fc5a84b8f000`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:46:26.966369 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-66023f2a-4287-479e-8885-a3e40ac5d6d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:46:27.052313 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:27.111604 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1129/2240] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:46:26 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:46:27.116033 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38be7287-205e-421a-810f-1e45d205d232 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:27.117525 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-38be7287-205e-421a-810f-1e45d205d232 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:46:27.207337 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38be7287-205e-421a-810f-1e45d205d232 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:27.215399 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec32299c-2f0b-48ea-a3ce-e99f2c4c12f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:27.216388 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec32299c-2f0b-48ea-a3ce-e99f2c4c12f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:27.216878 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec32299c-2f0b-48ea-a3ce-e99f2c4c12f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:27.217414 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec32299c-2f0b-48ea-a3ce-e99f2c4c12f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:27.219214 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec32299c-2f0b-48ea-a3ce-e99f2c4c12f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:27.221532 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec32299c-2f0b-48ea-a3ce-e99f2c4c12f2 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:27.252908 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1112/2241] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jun 16 17:46:27 2022] POST /identity/v3/projects => generated 385 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:46:27.257356 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66023f2a-4287-479e-8885-a3e40ac5d6d0 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:27.273021 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66023f2a-4287-479e-8885-a3e40ac5d6d0 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:27.345723 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6582b5-b6ad-4bc5-8f2a-e008b7dd02ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:27.346657 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6582b5-b6ad-4bc5-8f2a-e008b7dd02ff admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:27.347020 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6582b5-b6ad-4bc5-8f2a-e008b7dd02ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:27.347405 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6582b5-b6ad-4bc5-8f2a-e008b7dd02ff admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:27.348201 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6582b5-b6ad-4bc5-8f2a-e008b7dd02ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:27.350432 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6582b5-b6ad-4bc5-8f2a-e008b7dd02ff admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:27.373472 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1130/2242] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jun 16 17:46:27 2022] POST /identity/v3/users => generated 299 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:46:27.378221 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec32299c-2f0b-48ea-a3ce-e99f2c4c12f2 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:27.402098 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec32299c-2f0b-48ea-a3ce-e99f2c4c12f2 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:27.407926 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f9ed9f-af8f-45e2-92d2-740ad2561e60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:27.408633 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f9ed9f-af8f-45e2-92d2-740ad2561e60 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:27.409085 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f9ed9f-af8f-45e2-92d2-740ad2561e60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:27.409357 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f9ed9f-af8f-45e2-92d2-740ad2561e60 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:27.410243 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f9ed9f-af8f-45e2-92d2-740ad2561e60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:27.411855 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f9ed9f-af8f-45e2-92d2-740ad2561e60 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:27.426780 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1113/2243] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:46:27 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:46:27.430580 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6582b5-b6ad-4bc5-8f2a-e008b7dd02ff admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:27.442034 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6582b5-b6ad-4bc5-8f2a-e008b7dd02ff admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:27.446865 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60220a9c-464e-49f8-9601-cd7541e73176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:27.447767 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60220a9c-464e-49f8-9601-cd7541e73176 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:27.448051 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60220a9c-464e-49f8-9601-cd7541e73176 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:27.448331 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60220a9c-464e-49f8-9601-cd7541e73176 admin admin] PATH_INFO: `/v3/projects/929cd3d64af3439d9cd63858fcb8fc02/users/016d0bc441174f408ef50e5eced2e692/roles/3ca2dcbbd29d4a31a3ab78f2299bc239` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:27.470299 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60220a9c-464e-49f8-9601-cd7541e73176 admin admin] RBAC: Authorizing `identity:create_grant(project_id=929cd3d64af3439d9cd63858fcb8fc02, user_id=016d0bc441174f408ef50e5eced2e692, role_id=3ca2dcbbd29d4a31a3ab78f2299bc239)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:27.473019 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60220a9c-464e-49f8-9601-cd7541e73176 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:27.492846 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1131/2244] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:46:27 2022] PUT /identity/v3/projects/929cd3d64af3439d9cd63858fcb8fc02/users/016d0bc441174f408ef50e5eced2e692/roles/3ca2dcbbd29d4a31a3ab78f2299bc239 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:46:27.493747 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f9ed9f-af8f-45e2-92d2-740ad2561e60 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:27.505124 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f9ed9f-af8f-45e2-92d2-740ad2561e60 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:27.610705 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b16df4b-7417-467b-af50-374231d1a4db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:27.611395 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b16df4b-7417-467b-af50-374231d1a4db admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:27.611686 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b16df4b-7417-467b-af50-374231d1a4db admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:27.612049 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b16df4b-7417-467b-af50-374231d1a4db admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:27.612899 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b16df4b-7417-467b-af50-374231d1a4db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:27.614515 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b16df4b-7417-467b-af50-374231d1a4db admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:27.631786 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1114/2245] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:46:27 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 17:46:27.636453 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60220a9c-464e-49f8-9601-cd7541e73176 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:27.649265 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60220a9c-464e-49f8-9601-cd7541e73176 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:27.655878 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3e329a-9502-408e-aac6-08a5de2d2d95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:27.657087 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3e329a-9502-408e-aac6-08a5de2d2d95 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:27.657645 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3e329a-9502-408e-aac6-08a5de2d2d95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:27.657936 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3e329a-9502-408e-aac6-08a5de2d2d95 admin admin] PATH_INFO: `/v3/projects/929cd3d64af3439d9cd63858fcb8fc02/users/016d0bc441174f408ef50e5eced2e692/roles/3ca2dcbbd29d4a31a3ab78f2299bc239` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:27.660577 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3e329a-9502-408e-aac6-08a5de2d2d95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=929cd3d64af3439d9cd63858fcb8fc02, user_id=016d0bc441174f408ef50e5eced2e692, role_id=3ca2dcbbd29d4a31a3ab78f2299bc239)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:27.662984 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3e329a-9502-408e-aac6-08a5de2d2d95 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:27.676310 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1132/2246] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:46:27 2022] PUT /identity/v3/projects/929cd3d64af3439d9cd63858fcb8fc02/users/016d0bc441174f408ef50e5eced2e692/roles/3ca2dcbbd29d4a31a3ab78f2299bc239 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:46:33.719320 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2048a20-c4b5-44ca-b04a-e9c2fa882bc7 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:33.719912 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2048a20-c4b5-44ca-b04a-e9c2fa882bc7 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:33.719912 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2048a20-c4b5-44ca-b04a-e9c2fa882bc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:33.762354 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-a2048a20-c4b5-44ca-b04a-e9c2fa882bc7 None None] MFA Rules not processed for user `016d0bc441174f408ef50e5eced2e692`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:46:33.763219 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-a2048a20-c4b5-44ca-b04a-e9c2fa882bc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:46:33.819273 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:INSERT=2,keystone:DELETE=3,keystone:SELECT=95 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:33.866379 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1115/2247] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:46:33 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:46:33.876122 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3e329a-9502-408e-aac6-08a5de2d2d95 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:33.878382 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-2d3e329a-9502-408e-aac6-08a5de2d2d95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:46:33.997518 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3e329a-9502-408e-aac6-08a5de2d2d95 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:34.002992 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaae4c0d-58f3-4c2d-9518-99f0102ed840 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd89029c1b3134df1962c041988bead74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:34.003786 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaae4c0d-58f3-4c2d-9518-99f0102ed840 service glance] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:34.004833 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaae4c0d-58f3-4c2d-9518-99f0102ed840 service glance] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:34.005224 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaae4c0d-58f3-4c2d-9518-99f0102ed840 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:34.066712 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:34.098749 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaae4c0d-58f3-4c2d-9518-99f0102ed840 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:34.100264 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaae4c0d-58f3-4c2d-9518-99f0102ed840 service glance] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:34.132788 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1133/2248] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:46:33 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:46:34.302889 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb9fac4-b3ed-4dc9-ad1f-46044f8ed021 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:34.302889 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb9fac4-b3ed-4dc9-ad1f-46044f8ed021 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:34.302889 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb9fac4-b3ed-4dc9-ad1f-46044f8ed021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:34.349498 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-fdb9fac4-b3ed-4dc9-ad1f-46044f8ed021 None None] MFA Rules not processed for user `71443c30b4554719b5549493bf3cc00c`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:46:34.350931 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-fdb9fac4-b3ed-4dc9-ad1f-46044f8ed021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:46:34.485350 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1116/2249] 10.222.0.42 () {58 vars in 1009 bytes} [Thu Jun 16 17:46:34 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:46:34.554347 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaae4c0d-58f3-4c2d-9518-99f0102ed840 service glance] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:34.556329 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-eaae4c0d-58f3-4c2d-9518-99f0102ed840 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:46:34.654620 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaae4c0d-58f3-4c2d-9518-99f0102ed840 service glance] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:34.661178 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa17a25-0beb-4501-963b-21f6cde19360 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47b7a9cd615c41d6983a0007b475038c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:34.662688 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa17a25-0beb-4501-963b-21f6cde19360 service swift] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:34.662774 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa17a25-0beb-4501-963b-21f6cde19360 service swift] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:34.663278 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa17a25-0beb-4501-963b-21f6cde19360 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:34.676883 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa17a25-0beb-4501-963b-21f6cde19360 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:34.678936 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa17a25-0beb-4501-963b-21f6cde19360 service swift] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:34.694675 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1134/2250] 10.222.0.42 () {60 vars in 1384 bytes} [Thu Jun 16 17:46:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 17:46:37.075773 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18391fb3-e69e-4d83-8824-9fc465822dbe None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:37.076814 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18391fb3-e69e-4d83-8824-9fc465822dbe None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:37.077511 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18391fb3-e69e-4d83-8824-9fc465822dbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:37.100985 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-18391fb3-e69e-4d83-8824-9fc465822dbe None None] MFA Rules not processed for user `71443c30b4554719b5549493bf3cc00c`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:46:37.102411 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-18391fb3-e69e-4d83-8824-9fc465822dbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:46:37.119164 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1117/2251] 10.222.0.42 () {58 vars in 1009 bytes} [Thu Jun 16 17:46:37 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:46:37.137438 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa17a25-0beb-4501-963b-21f6cde19360 service swift] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:37.149557 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa17a25-0beb-4501-963b-21f6cde19360 service swift] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:37.155498 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933b9ab4-132d-44b4-9d17-02b919ce6d33 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47b7a9cd615c41d6983a0007b475038c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:37.156165 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933b9ab4-132d-44b4-9d17-02b919ce6d33 service swift] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:37.156504 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933b9ab4-132d-44b4-9d17-02b919ce6d33 service swift] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:37.156807 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933b9ab4-132d-44b4-9d17-02b919ce6d33 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:37.166310 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933b9ab4-132d-44b4-9d17-02b919ce6d33 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:37.168156 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933b9ab4-132d-44b4-9d17-02b919ce6d33 service swift] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:37.178806 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1135/2252] 10.222.0.42 () {60 vars in 1384 bytes} [Thu Jun 16 17:46:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 17:46:37.625880 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3678be-ac03-4e46-acb8-788227987034 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:37.626559 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3678be-ac03-4e46-acb8-788227987034 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:37.627078 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3678be-ac03-4e46-acb8-788227987034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:37.653886 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-1f3678be-ac03-4e46-acb8-788227987034 None None] MFA Rules not processed for user `71443c30b4554719b5549493bf3cc00c`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:46:37.656230 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-1f3678be-ac03-4e46-acb8-788227987034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:46:37.665503 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1118/2253] 10.222.0.42 () {58 vars in 1009 bytes} [Thu Jun 16 17:46:37 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:46:37.683668 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933b9ab4-132d-44b4-9d17-02b919ce6d33 service swift] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:37.697992 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933b9ab4-132d-44b4-9d17-02b919ce6d33 service swift] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:37.707162 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f72882-ae8c-449d-8084-d6c6cd1c25d6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47b7a9cd615c41d6983a0007b475038c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:37.707956 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f72882-ae8c-449d-8084-d6c6cd1c25d6 service swift] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:37.708242 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f72882-ae8c-449d-8084-d6c6cd1c25d6 service swift] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:37.708976 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f72882-ae8c-449d-8084-d6c6cd1c25d6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:37.726006 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f72882-ae8c-449d-8084-d6c6cd1c25d6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:37.727913 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f72882-ae8c-449d-8084-d6c6cd1c25d6 service swift] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:37.739744 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1136/2254] 10.222.0.42 () {60 vars in 1384 bytes} [Thu Jun 16 17:46:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 17:46:38.626450 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02afd5ca-1eb9-4001-9a61-9317693b95be None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:38.627396 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02afd5ca-1eb9-4001-9a61-9317693b95be None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:38.627877 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02afd5ca-1eb9-4001-9a61-9317693b95be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:38.676784 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-02afd5ca-1eb9-4001-9a61-9317693b95be None None] MFA Rules not processed for user `8fff53cb0d084d9284825488f6843ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:46:38.678374 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-02afd5ca-1eb9-4001-9a61-9317693b95be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:46:38.799614 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f64362-4135-4457-94d3-820aba4b83e7 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:38.800193 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f64362-4135-4457-94d3-820aba4b83e7 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:38.800193 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f64362-4135-4457-94d3-820aba4b83e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:38.815073 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1119/2255] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:46:38 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:46:38.847974 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-c3f64362-4135-4457-94d3-820aba4b83e7 None None] MFA Rules not processed for user `268c5947a265492bb7a8f181f9f5c504`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:46:38.849277 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-c3f64362-4135-4457-94d3-820aba4b83e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:46:39.023166 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1137/2256] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:46:38 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:46:46.711227 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2f24e1-0958-4eb5-9828-122ef22a5fdc None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:46.712904 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2f24e1-0958-4eb5-9828-122ef22a5fdc None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:46.719081 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2f24e1-0958-4eb5-9828-122ef22a5fdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:46.802068 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-df2f24e1-0958-4eb5-9828-122ef22a5fdc None None] MFA Rules not processed for user `71443c30b4554719b5549493bf3cc00c`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:46:46.803646 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-df2f24e1-0958-4eb5-9828-122ef22a5fdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:46:46.833635 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1120/2257] 10.222.0.42 () {58 vars in 967 bytes} [Thu Jun 16 17:46:46 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:46:46.876334 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f64362-4135-4457-94d3-820aba4b83e7 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:46.925722 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f64362-4135-4457-94d3-820aba4b83e7 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:46.934063 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdb6d54-66d4-4847-992e-9c26b52c1b68 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47b7a9cd615c41d6983a0007b475038c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:46.934719 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdb6d54-66d4-4847-992e-9c26b52c1b68 service swift] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:46.935007 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdb6d54-66d4-4847-992e-9c26b52c1b68 service swift] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:46.935323 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdb6d54-66d4-4847-992e-9c26b52c1b68 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:46.955599 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdb6d54-66d4-4847-992e-9c26b52c1b68 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:46.957284 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdb6d54-66d4-4847-992e-9c26b52c1b68 service swift] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:46.967470 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1138/2258] 10.222.0.42 () {60 vars in 1384 bytes} [Thu Jun 16 17:46:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 17:46:47.320518 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bf66ad-c998-4c32-9b89-9c702258036e None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:47.320719 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bf66ad-c998-4c32-9b89-9c702258036e None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:47.320962 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bf66ad-c998-4c32-9b89-9c702258036e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:47.366053 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-e6bf66ad-c998-4c32-9b89-9c702258036e None None] MFA Rules not processed for user `71443c30b4554719b5549493bf3cc00c`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:46:47.367517 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-e6bf66ad-c998-4c32-9b89-9c702258036e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:46:47.390339 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1121/2259] 10.222.0.42 () {58 vars in 1009 bytes} [Thu Jun 16 17:46:47 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:46:47.410349 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdb6d54-66d4-4847-992e-9c26b52c1b68 service swift] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:47.427718 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdb6d54-66d4-4847-992e-9c26b52c1b68 service swift] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:47.444609 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc4f006-7625-4b64-8248-694bb3edf8c3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47b7a9cd615c41d6983a0007b475038c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:47.445429 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc4f006-7625-4b64-8248-694bb3edf8c3 service swift] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:47.445721 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc4f006-7625-4b64-8248-694bb3edf8c3 service swift] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:47.446006 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc4f006-7625-4b64-8248-694bb3edf8c3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:47.466910 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc4f006-7625-4b64-8248-694bb3edf8c3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:47.470031 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc4f006-7625-4b64-8248-694bb3edf8c3 service swift] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:47.517967 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1139/2260] 10.222.0.42 () {60 vars in 1384 bytes} [Thu Jun 16 17:46:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 17:46:47.920710 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeeb38d-a211-45d4-97c2-f97458da0969 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:47.921147 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeeb38d-a211-45d4-97c2-f97458da0969 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:47.924253 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeeb38d-a211-45d4-97c2-f97458da0969 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:47.963919 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-1eeeb38d-a211-45d4-97c2-f97458da0969 None None] MFA Rules not processed for user `03be2d45633949a7b416fc5a84b8f000`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:46:47.965234 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-1eeeb38d-a211-45d4-97c2-f97458da0969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:46:47.986130 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1122/2261] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:46:47 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:46:48.535268 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c0c7f3-5a68-41ba-994a-c3294a196dec None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:48.535787 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c0c7f3-5a68-41ba-994a-c3294a196dec None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:48.535870 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c0c7f3-5a68-41ba-994a-c3294a196dec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:48.555606 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-a1c0c7f3-5a68-41ba-994a-c3294a196dec None None] MFA Rules not processed for user `71443c30b4554719b5549493bf3cc00c`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:46:48.557077 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-a1c0c7f3-5a68-41ba-994a-c3294a196dec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:46:48.566007 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1140/2262] 10.222.0.42 () {58 vars in 967 bytes} [Thu Jun 16 17:46:48 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:46:48.580086 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eeeb38d-a211-45d4-97c2-f97458da0969 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:48.587032 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=100 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:48.642127 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eeeb38d-a211-45d4-97c2-f97458da0969 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:48.649514 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de816205-c5b2-41b7-bee2-ab01bb573b09 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47b7a9cd615c41d6983a0007b475038c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:48.650445 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de816205-c5b2-41b7-bee2-ab01bb573b09 service swift] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:48.651055 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de816205-c5b2-41b7-bee2-ab01bb573b09 service swift] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:48.651476 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de816205-c5b2-41b7-bee2-ab01bb573b09 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:48.677401 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de816205-c5b2-41b7-bee2-ab01bb573b09 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:48.679301 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de816205-c5b2-41b7-bee2-ab01bb573b09 service swift] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:48.698133 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1123/2263] 10.222.0.42 () {60 vars in 1384 bytes} [Thu Jun 16 17:46:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 17:46:51.610339 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c0c7f3-5a68-41ba-994a-c3294a196dec None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:51.612123 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-a1c0c7f3-5a68-41ba-994a-c3294a196dec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:46:51.621976 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:51.687298 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c0c7f3-5a68-41ba-994a-c3294a196dec None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:51.692779 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4b1888-e5f1-47b3-9f97-1da1f8ab1a29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:51.693730 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4b1888-e5f1-47b3-9f97-1da1f8ab1a29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:51.694077 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4b1888-e5f1-47b3-9f97-1da1f8ab1a29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:51.694419 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4b1888-e5f1-47b3-9f97-1da1f8ab1a29 admin admin] PATH_INFO: `/v3/users/7e7032bdea794e57a7d6d1ad1cd2439e` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:51.695731 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4b1888-e5f1-47b3-9f97-1da1f8ab1a29 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e7032bdea794e57a7d6d1ad1cd2439e)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:51.697839 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4b1888-e5f1-47b3-9f97-1da1f8ab1a29 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:51.777454 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-0c4b1888-e5f1-47b3-9f97-1da1f8ab1a29 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e7032bdea794e57a7d6d1ad1cd2439e'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:46:51.793022 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-0c4b1888-e5f1-47b3-9f97-1da1f8ab1a29 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e7032bdea794e57a7d6d1ad1cd2439e'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:46:51.793682 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-0c4b1888-e5f1-47b3-9f97-1da1f8ab1a29 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e7032bdea794e57a7d6d1ad1cd2439e'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:46:51.803755 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-0c4b1888-e5f1-47b3-9f97-1da1f8ab1a29 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e7032bdea794e57a7d6d1ad1cd2439e'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:46:51.814209 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-0c4b1888-e5f1-47b3-9f97-1da1f8ab1a29 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e7032bdea794e57a7d6d1ad1cd2439e'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:46:51.820890 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1141/2264] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jun 16 17:46:51 2022] DELETE /identity/v3/users/7e7032bdea794e57a7d6d1ad1cd2439e => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:46:51.825438 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813102f2-e8b9-4047-8c7c-3877fbe8b8b8 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:51.826828 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813102f2-e8b9-4047-8c7c-3877fbe8b8b8 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:51.827122 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813102f2-e8b9-4047-8c7c-3877fbe8b8b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:51.845772 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-813102f2-e8b9-4047-8c7c-3877fbe8b8b8 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:46:51.847196 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-813102f2-e8b9-4047-8c7c-3877fbe8b8b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:46:51.948845 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1124/2265] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:46:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:46:51.966095 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4b1888-e5f1-47b3-9f97-1da1f8ab1a29 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:51.968278 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-0c4b1888-e5f1-47b3-9f97-1da1f8ab1a29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:46:52.055745 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4b1888-e5f1-47b3-9f97-1da1f8ab1a29 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:52.062295 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dd5658-7ae0-4d11-9dd0-23bf107f9e0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:52.062965 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dd5658-7ae0-4d11-9dd0-23bf107f9e0d service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:52.063297 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dd5658-7ae0-4d11-9dd0-23bf107f9e0d service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:52.063635 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dd5658-7ae0-4d11-9dd0-23bf107f9e0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:52.075010 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dd5658-7ae0-4d11-9dd0-23bf107f9e0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:52.077944 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dd5658-7ae0-4d11-9dd0-23bf107f9e0d service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:52.091869 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1142/2266] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:46:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:46:52.471819 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813102f2-e8b9-4047-8c7c-3877fbe8b8b8 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:52.473808 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-813102f2-e8b9-4047-8c7c-3877fbe8b8b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:46:52.491014 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813102f2-e8b9-4047-8c7c-3877fbe8b8b8 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:52.502567 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b6b267-2f85-4beb-a273-d119a3428a43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:52.503639 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b6b267-2f85-4beb-a273-d119a3428a43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:52.504064 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b6b267-2f85-4beb-a273-d119a3428a43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:52.504424 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b6b267-2f85-4beb-a273-d119a3428a43 admin admin] PATH_INFO: `/v3/projects/bb35ec3efb8c4a35b70191ebacce35a9` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:52.506378 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b6b267-2f85-4beb-a273-d119a3428a43 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb35ec3efb8c4a35b70191ebacce35a9)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:52.508701 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b6b267-2f85-4beb-a273-d119a3428a43 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:52.583705 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-a0b6b267-2f85-4beb-a273-d119a3428a43 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb35ec3efb8c4a35b70191ebacce35a9', 'request_id': 'req-a0b6b267-2f85-4beb-a273-d119a3428a43'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:46:52.594856 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-a0b6b267-2f85-4beb-a273-d119a3428a43 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb35ec3efb8c4a35b70191ebacce35a9', 'request_id': 'req-a0b6b267-2f85-4beb-a273-d119a3428a43'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:46:52.602417 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-a0b6b267-2f85-4beb-a273-d119a3428a43 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb35ec3efb8c4a35b70191ebacce35a9', 'request_id': 'req-a0b6b267-2f85-4beb-a273-d119a3428a43'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:46:52.620787 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-a0b6b267-2f85-4beb-a273-d119a3428a43 admin admin] The token cache is being invalidate because project bb35ec3efb8c4a35b70191ebacce35a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80357) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:46:52.621193 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-a0b6b267-2f85-4beb-a273-d119a3428a43 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:46:52.622814 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-a0b6b267-2f85-4beb-a273-d119a3428a43 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:46:52.625763 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1125/2267] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jun 16 17:46:52 2022] DELETE /identity/v3/projects/bb35ec3efb8c4a35b70191ebacce35a9 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:46:53.285971 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200499fb-08cb-422e-9843-3b859b952ff6 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:53.286862 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200499fb-08cb-422e-9843-3b859b952ff6 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:53.287351 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200499fb-08cb-422e-9843-3b859b952ff6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:53.311395 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-200499fb-08cb-422e-9843-3b859b952ff6 None None] MFA Rules not processed for user `71443c30b4554719b5549493bf3cc00c`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:46:53.312665 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-200499fb-08cb-422e-9843-3b859b952ff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:46:53.337636 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:53.374443 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1143/2268] 10.222.0.42 () {58 vars in 1009 bytes} [Thu Jun 16 17:46:53 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:46:53.388307 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b6b267-2f85-4beb-a273-d119a3428a43 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:53.390528 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-a0b6b267-2f85-4beb-a273-d119a3428a43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:46:53.485044 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:53.489847 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b6b267-2f85-4beb-a273-d119a3428a43 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:53.497352 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35ddd2b-e186-4799-aecc-9fe5fd55ae21 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47b7a9cd615c41d6983a0007b475038c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:53.498032 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35ddd2b-e186-4799-aecc-9fe5fd55ae21 service swift] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:53.499586 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35ddd2b-e186-4799-aecc-9fe5fd55ae21 service swift] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:53.500229 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35ddd2b-e186-4799-aecc-9fe5fd55ae21 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:53.523267 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35ddd2b-e186-4799-aecc-9fe5fd55ae21 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:53.524852 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35ddd2b-e186-4799-aecc-9fe5fd55ae21 service swift] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:53.534185 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1126/2269] 10.222.0.42 () {60 vars in 1384 bytes} [Thu Jun 16 17:46:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 17:47:03.366579 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=12 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:03.530917 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=4 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:04.063536 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200499fb-08cb-422e-9843-3b859b952ff6 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:04.065080 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-200499fb-08cb-422e-9843-3b859b952ff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:47:04.169121 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200499fb-08cb-422e-9843-3b859b952ff6 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:04.174830 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34a868d-b0be-4b80-a68b-4de132b2fea6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0570d8d9508d4546ae897276f7d181d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:04.175488 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34a868d-b0be-4b80-a68b-4de132b2fea6 service placement] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:04.175776 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34a868d-b0be-4b80-a68b-4de132b2fea6 service placement] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:04.176054 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34a868d-b0be-4b80-a68b-4de132b2fea6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:04.177916 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-b34a868d-b0be-4b80-a68b-4de132b2fea6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:47:04.189777 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34a868d-b0be-4b80-a68b-4de132b2fea6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:04.191392 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34a868d-b0be-4b80-a68b-4de132b2fea6 service placement] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:04.201722 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1144/2270] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:47:04 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:47:07.086639 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaa204e-76c0-4561-b443-c8dcaf776379 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:07.087150 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaa204e-76c0-4561-b443-c8dcaf776379 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:07.087253 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaa204e-76c0-4561-b443-c8dcaf776379 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:07.088794 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1127/2271] 10.222.0.42 () {54 vars in 921 bytes} [Thu Jun 16 17:47:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:47:07.101200 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df875b4-b642-499f-85db-300238f2082f None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:07.101691 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df875b4-b642-499f-85db-300238f2082f None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:07.101796 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df875b4-b642-499f-85db-300238f2082f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:07.188925 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-5df875b4-b642-499f-85db-300238f2082f None None] MFA Rules not processed for user `f86c16179a70474a817be9fda33ece9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:47:07.190806 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-5df875b4-b642-499f-85db-300238f2082f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:47:07.289601 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1145/2272] 10.222.0.42 () {58 vars in 1018 bytes} [Thu Jun 16 17:47:07 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:47:07.302606 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaa204e-76c0-4561-b443-c8dcaf776379 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:07.304614 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-ffaa204e-76c0-4561-b443-c8dcaf776379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:47:07.409119 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaa204e-76c0-4561-b443-c8dcaf776379 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:07.413088 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b56575-7aae-46b7-bce9-998b43462dc1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd89029c1b3134df1962c041988bead74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:07.413754 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b56575-7aae-46b7-bce9-998b43462dc1 service glance] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:07.414031 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b56575-7aae-46b7-bce9-998b43462dc1 service glance] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:07.414307 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b56575-7aae-46b7-bce9-998b43462dc1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:07.426392 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b56575-7aae-46b7-bce9-998b43462dc1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:07.427853 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b56575-7aae-46b7-bce9-998b43462dc1 service glance] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:07.440835 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1128/2273] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:47:07 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:47:07.512960 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdc46e5-f242-43ae-8297-140a5a640881 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:07.513809 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdc46e5-f242-43ae-8297-140a5a640881 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:07.514071 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdc46e5-f242-43ae-8297-140a5a640881 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:07.515444 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1146/2274] 10.222.0.42 () {54 vars in 921 bytes} [Thu Jun 16 17:47:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:47:07.520786 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef35879-f38b-4a7e-b5ba-93ca6973fa99 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:07.521082 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef35879-f38b-4a7e-b5ba-93ca6973fa99 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:07.521367 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef35879-f38b-4a7e-b5ba-93ca6973fa99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:07.541343 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-aef35879-f38b-4a7e-b5ba-93ca6973fa99 None None] MFA Rules not processed for user `f94416103f6245888214a997b014f47a`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:47:07.542583 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-aef35879-f38b-4a7e-b5ba-93ca6973fa99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:47:07.596139 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1129/2275] 10.222.0.42 () {58 vars in 1018 bytes} [Thu Jun 16 17:47:07 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:47:07.608530 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdc46e5-f242-43ae-8297-140a5a640881 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:07.610250 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-afdc46e5-f242-43ae-8297-140a5a640881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:47:07.623565 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdc46e5-f242-43ae-8297-140a5a640881 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:07.627067 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc7600a-7b6a-4921-81f1-37921189144d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:07.627712 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc7600a-7b6a-4921-81f1-37921189144d service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:07.627990 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc7600a-7b6a-4921-81f1-37921189144d service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:07.628260 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc7600a-7b6a-4921-81f1-37921189144d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:07.636155 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc7600a-7b6a-4921-81f1-37921189144d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:07.637627 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc7600a-7b6a-4921-81f1-37921189144d service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:07.647192 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1147/2276] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:47:07 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:47:07.735173 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b91e67c-6741-4a1e-984f-d6e5bb1c5f7d None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:07.735600 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b91e67c-6741-4a1e-984f-d6e5bb1c5f7d None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:07.736015 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b91e67c-6741-4a1e-984f-d6e5bb1c5f7d None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:07.738167 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1130/2277] 10.222.0.42 () {54 vars in 921 bytes} [Thu Jun 16 17:47:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:47:07.745724 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80969786-adcc-457f-af57-8ebd79566709 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:07.746104 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80969786-adcc-457f-af57-8ebd79566709 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:07.746448 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80969786-adcc-457f-af57-8ebd79566709 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:07.767998 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-80969786-adcc-457f-af57-8ebd79566709 None None] MFA Rules not processed for user `f86c16179a70474a817be9fda33ece9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:47:07.769393 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-80969786-adcc-457f-af57-8ebd79566709 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:47:07.784350 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1148/2278] 10.222.0.42 () {58 vars in 1018 bytes} [Thu Jun 16 17:47:07 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:47:07.795372 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b91e67c-6741-4a1e-984f-d6e5bb1c5f7d None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:07.807490 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b91e67c-6741-4a1e-984f-d6e5bb1c5f7d None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:07.812513 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11865165-08a1-43f7-a039-f0f0c411137d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:07.813231 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11865165-08a1-43f7-a039-f0f0c411137d service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:07.813555 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11865165-08a1-43f7-a039-f0f0c411137d service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:07.813833 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11865165-08a1-43f7-a039-f0f0c411137d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:07.824513 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11865165-08a1-43f7-a039-f0f0c411137d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:07.825938 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11865165-08a1-43f7-a039-f0f0c411137d service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:07.838675 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1131/2279] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:47:07 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:47:14.923943 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bd0223-2689-4f77-b37c-407a65f24018 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:14.924560 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bd0223-2689-4f77-b37c-407a65f24018 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:14.924560 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bd0223-2689-4f77-b37c-407a65f24018 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:14.944845 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-e6bd0223-2689-4f77-b37c-407a65f24018 None None] MFA Rules not processed for user `71443c30b4554719b5549493bf3cc00c`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:47:14.945710 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-e6bd0223-2689-4f77-b37c-407a65f24018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:47:14.956142 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1149/2280] 10.222.0.42 () {58 vars in 1009 bytes} [Thu Jun 16 17:47:14 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:47:14.967763 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11865165-08a1-43f7-a039-f0f0c411137d service neutron] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:14.979008 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11865165-08a1-43f7-a039-f0f0c411137d service neutron] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:14.984701 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65d07e6-4c7a-4929-86a2-5e5c9185090c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47b7a9cd615c41d6983a0007b475038c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:14.985340 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65d07e6-4c7a-4929-86a2-5e5c9185090c service swift] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:14.985625 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65d07e6-4c7a-4929-86a2-5e5c9185090c service swift] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:14.985901 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65d07e6-4c7a-4929-86a2-5e5c9185090c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:14.994330 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65d07e6-4c7a-4929-86a2-5e5c9185090c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:14.995925 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65d07e6-4c7a-4929-86a2-5e5c9185090c service swift] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:15.004860 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1132/2281] 10.222.0.42 () {60 vars in 1384 bytes} [Thu Jun 16 17:47:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 17:47:24.939913 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:25.001729 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=70 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:38.059510 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79a0788-15c3-4440-b594-276d8dca9639 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:38.059983 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79a0788-15c3-4440-b594-276d8dca9639 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:38.060125 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79a0788-15c3-4440-b594-276d8dca9639 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:38.087186 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-c79a0788-15c3-4440-b594-276d8dca9639 None None] MFA Rules not processed for user `71443c30b4554719b5549493bf3cc00c`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:47:38.088441 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-c79a0788-15c3-4440-b594-276d8dca9639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:47:38.096473 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1150/2282] 10.222.0.42 () {58 vars in 967 bytes} [Thu Jun 16 17:47:38 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:47:38.122793 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65d07e6-4c7a-4929-86a2-5e5c9185090c service swift] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:38.136815 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65d07e6-4c7a-4929-86a2-5e5c9185090c service swift] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:38.143674 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c11eb3-ba46-4da4-b26d-e2e6fc603713 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47b7a9cd615c41d6983a0007b475038c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:38.144356 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c11eb3-ba46-4da4-b26d-e2e6fc603713 service swift] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:38.144677 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c11eb3-ba46-4da4-b26d-e2e6fc603713 service swift] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:38.144956 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c11eb3-ba46-4da4-b26d-e2e6fc603713 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:38.154019 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c11eb3-ba46-4da4-b26d-e2e6fc603713 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:38.156229 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c11eb3-ba46-4da4-b26d-e2e6fc603713 service swift] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:38.167607 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1133/2283] 10.222.0.42 () {60 vars in 1384 bytes} [Thu Jun 16 17:47:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 17:47:40.010851 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79a0788-15c3-4440-b594-276d8dca9639 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:40.012262 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-c79a0788-15c3-4440-b594-276d8dca9639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:47:40.023783 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79a0788-15c3-4440-b594-276d8dca9639 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:40.027372 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ea749b-4529-4a25-a48e-e72f5f858d09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:40.028007 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ea749b-4529-4a25-a48e-e72f5f858d09 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:40.028293 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ea749b-4529-4a25-a48e-e72f5f858d09 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:40.028602 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ea749b-4529-4a25-a48e-e72f5f858d09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:40.030299 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-d7ea749b-4529-4a25-a48e-e72f5f858d09 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:47:40.093005 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ea749b-4529-4a25-a48e-e72f5f858d09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:40.094926 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ea749b-4529-4a25-a48e-e72f5f858d09 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:40.104653 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1151/2284] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:47:40 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:47:48.162905 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=6 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:50.100282 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=33 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:25.941520 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c93adc-eae0-47f1-9b70-ac7c2fd3b8ad None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:25.942075 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c93adc-eae0-47f1-9b70-ac7c2fd3b8ad None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:25.942075 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c93adc-eae0-47f1-9b70-ac7c2fd3b8ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:25.961012 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-b1c93adc-eae0-47f1-9b70-ac7c2fd3b8ad None None] MFA Rules not processed for user `8fff53cb0d084d9284825488f6843ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:48:25.962205 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-b1c93adc-eae0-47f1-9b70-ac7c2fd3b8ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:48:26.019423 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1134/2285] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:48:25 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:48:36.012091 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=25 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:38.024222 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04363d81-3eb5-45c6-a796-04ebb36b0175 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:38.025108 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04363d81-3eb5-45c6-a796-04ebb36b0175 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:38.025392 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04363d81-3eb5-45c6-a796-04ebb36b0175 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:38.043318 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-04363d81-3eb5-45c6-a796-04ebb36b0175 None None] MFA Rules not processed for user `7b078c6bfca04de790e7b2e45739033e`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:48:38.044551 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-04363d81-3eb5-45c6-a796-04ebb36b0175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:48:38.099603 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1152/2286] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:48:38 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:48:45.109803 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8ab0df-1441-4b89-af13-e9944eaf59a5 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:45.110848 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8ab0df-1441-4b89-af13-e9944eaf59a5 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:45.111001 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8ab0df-1441-4b89-af13-e9944eaf59a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:45.128238 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-5e8ab0df-1441-4b89-af13-e9944eaf59a5 None None] MFA Rules not processed for user `268c5947a265492bb7a8f181f9f5c504`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:48:45.129499 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-5e8ab0df-1441-4b89-af13-e9944eaf59a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:48:45.186879 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1135/2287] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:48:45 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:48:48.091433 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=25 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:50.978598 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04363d81-3eb5-45c6-a796-04ebb36b0175 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:50.988255 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04363d81-3eb5-45c6-a796-04ebb36b0175 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:50.992711 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b330322-2ed8-40e9-b464-6501ce001cab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:50.993397 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b330322-2ed8-40e9-b464-6501ce001cab service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:50.993737 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b330322-2ed8-40e9-b464-6501ce001cab service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:50.994013 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b330322-2ed8-40e9-b464-6501ce001cab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:50.995848 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-6b330322-2ed8-40e9-b464-6501ce001cab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:48:51.010580 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b330322-2ed8-40e9-b464-6501ce001cab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:51.012046 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b330322-2ed8-40e9-b464-6501ce001cab service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:51.024061 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1153/2288] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:48:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:48:53.898377 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8ab0df-1441-4b89-af13-e9944eaf59a5 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:53.915469 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8ab0df-1441-4b89-af13-e9944eaf59a5 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:53.920143 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fe0e8b-a206-456a-b965-8efef0e54477 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:53.921092 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fe0e8b-a206-456a-b965-8efef0e54477 service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:53.921768 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fe0e8b-a206-456a-b965-8efef0e54477 service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:53.922118 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fe0e8b-a206-456a-b965-8efef0e54477 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:53.927049 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-68fe0e8b-a206-456a-b965-8efef0e54477 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:48:53.970181 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fe0e8b-a206-456a-b965-8efef0e54477 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:53.972793 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fe0e8b-a206-456a-b965-8efef0e54477 service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:53.988917 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1136/2289] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:48:53 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:48:54.866719 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf04f18b-0034-420b-b6c1-510da2da8f0c None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:54.867323 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf04f18b-0034-420b-b6c1-510da2da8f0c None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:54.867323 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf04f18b-0034-420b-b6c1-510da2da8f0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:54.885657 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-cf04f18b-0034-420b-b6c1-510da2da8f0c None None] MFA Rules not processed for user `813dd0cf7e1d4724b903b358aaac1ea7`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:48:54.886953 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-cf04f18b-0034-420b-b6c1-510da2da8f0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:48:54.942609 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1154/2290] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:48:54 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:49:01.665750 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fe0e8b-a206-456a-b965-8efef0e54477 service neutron] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:49:01.667262 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-68fe0e8b-a206-456a-b965-8efef0e54477 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:49:01.682208 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fe0e8b-a206-456a-b965-8efef0e54477 service neutron] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:49:01.687747 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e456bfaa-74c1-47ad-872a-ba27a0885efe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f86c16179a70474a817be9fda33ece9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:49:01.689286 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e456bfaa-74c1-47ad-872a-ba27a0885efe service nova] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:49:01.689485 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e456bfaa-74c1-47ad-872a-ba27a0885efe service nova] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:49:01.689729 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e456bfaa-74c1-47ad-872a-ba27a0885efe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:49:01.691584 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-e456bfaa-74c1-47ad-872a-ba27a0885efe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:49:01.702316 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e456bfaa-74c1-47ad-872a-ba27a0885efe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:49:01.704359 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e456bfaa-74c1-47ad-872a-ba27a0885efe service nova] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:49:01.714277 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1137/2291] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:49:01 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:49:04.935171 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=31 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:49:11.709596 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=37 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:49:25.311392 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf04f18b-0034-420b-b6c1-510da2da8f0c None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:49:25.312727 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-cf04f18b-0034-420b-b6c1-510da2da8f0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:49:25.328299 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf04f18b-0034-420b-b6c1-510da2da8f0c None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:49:25.334240 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655abdb0-c4f8-496f-95a7-1e6323bc5175 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f86c16179a70474a817be9fda33ece9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:49:25.334892 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655abdb0-c4f8-496f-95a7-1e6323bc5175 service nova] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:49:25.335177 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655abdb0-c4f8-496f-95a7-1e6323bc5175 service nova] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:49:25.335458 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655abdb0-c4f8-496f-95a7-1e6323bc5175 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:49:25.337256 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-655abdb0-c4f8-496f-95a7-1e6323bc5175 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:49:25.348227 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655abdb0-c4f8-496f-95a7-1e6323bc5175 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:49:25.349844 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655abdb0-c4f8-496f-95a7-1e6323bc5175 service nova] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:49:25.358767 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1155/2292] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:49:25 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:49:29.476549 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef10fd48-ba2d-43da-b432-a4ac01435e59 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:49:29.477019 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef10fd48-ba2d-43da-b432-a4ac01435e59 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:49:29.477139 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef10fd48-ba2d-43da-b432-a4ac01435e59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:49:29.495103 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-ef10fd48-ba2d-43da-b432-a4ac01435e59 None None] MFA Rules not processed for user `71443c30b4554719b5549493bf3cc00c`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:49:29.496294 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-ef10fd48-ba2d-43da-b432-a4ac01435e59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:49:29.504071 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1138/2293] 10.222.0.42 () {58 vars in 967 bytes} [Thu Jun 16 17:49:29 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:49:29.551289 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655abdb0-c4f8-496f-95a7-1e6323bc5175 service nova] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:49:29.562452 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655abdb0-c4f8-496f-95a7-1e6323bc5175 service nova] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:49:29.568358 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b617bbfc-e370-4c5e-b718-5fa425261790 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47b7a9cd615c41d6983a0007b475038c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:49:29.569018 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b617bbfc-e370-4c5e-b718-5fa425261790 service swift] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:49:29.569302 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b617bbfc-e370-4c5e-b718-5fa425261790 service swift] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:49:29.569580 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b617bbfc-e370-4c5e-b718-5fa425261790 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:49:29.577686 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b617bbfc-e370-4c5e-b718-5fa425261790 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:49:29.579265 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b617bbfc-e370-4c5e-b718-5fa425261790 service swift] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:49:29.587372 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1156/2294] 10.222.0.42 () {60 vars in 1384 bytes} [Thu Jun 16 17:49:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 17:49:34.875789 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef10fd48-ba2d-43da-b432-a4ac01435e59 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:49:34.878077 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-ef10fd48-ba2d-43da-b432-a4ac01435e59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:49:34.989804 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef10fd48-ba2d-43da-b432-a4ac01435e59 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:49:34.993681 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed581288-9e59-4e95-89a9-8d43cc9f6788 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24d5e2637b7043e08313909d5b06e329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:49:34.994689 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed581288-9e59-4e95-89a9-8d43cc9f6788 service cinder] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:49:34.994975 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed581288-9e59-4e95-89a9-8d43cc9f6788 service cinder] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:49:34.995240 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed581288-9e59-4e95-89a9-8d43cc9f6788 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:49:34.997275 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-ed581288-9e59-4e95-89a9-8d43cc9f6788 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:49:35.011734 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed581288-9e59-4e95-89a9-8d43cc9f6788 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:49:35.013725 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed581288-9e59-4e95-89a9-8d43cc9f6788 service cinder] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:49:35.028980 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1139/2295] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:49:34 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:49:36.651897 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b617bbfc-e370-4c5e-b718-5fa425261790 service swift] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:49:36.653434 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-b617bbfc-e370-4c5e-b718-5fa425261790 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:49:36.721335 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b617bbfc-e370-4c5e-b718-5fa425261790 service swift] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:49:36.725973 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6273d30-f45c-43b1-994d-e7bc4fa210ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:49:36.726842 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6273d30-f45c-43b1-994d-e7bc4fa210ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:49:36.727125 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6273d30-f45c-43b1-994d-e7bc4fa210ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:49:36.727401 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6273d30-f45c-43b1-994d-e7bc4fa210ee admin admin] PATH_INFO: `/v3/users/03be2d45633949a7b416fc5a84b8f000` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:49:36.728666 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6273d30-f45c-43b1-994d-e7bc4fa210ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=03be2d45633949a7b416fc5a84b8f000)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:49:36.730544 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6273d30-f45c-43b1-994d-e7bc4fa210ee admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:49:36.776518 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-d6273d30-f45c-43b1-994d-e7bc4fa210ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03be2d45633949a7b416fc5a84b8f000'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:49:36.793137 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-d6273d30-f45c-43b1-994d-e7bc4fa210ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03be2d45633949a7b416fc5a84b8f000'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:49:36.793890 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-d6273d30-f45c-43b1-994d-e7bc4fa210ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03be2d45633949a7b416fc5a84b8f000'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:49:36.801596 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-d6273d30-f45c-43b1-994d-e7bc4fa210ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03be2d45633949a7b416fc5a84b8f000'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:49:36.808787 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-d6273d30-f45c-43b1-994d-e7bc4fa210ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03be2d45633949a7b416fc5a84b8f000'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:49:36.815373 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1157/2296] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jun 16 17:49:36 2022] DELETE /identity/v3/users/03be2d45633949a7b416fc5a84b8f000 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:49:37.008728 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed581288-9e59-4e95-89a9-8d43cc9f6788 service cinder] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:49:37.026459 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed581288-9e59-4e95-89a9-8d43cc9f6788 service cinder] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:49:37.089873 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27210c8-d99b-4394-94d7-3dc1337a4ac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:49:37.090749 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27210c8-d99b-4394-94d7-3dc1337a4ac2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:49:37.091045 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27210c8-d99b-4394-94d7-3dc1337a4ac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:49:37.091315 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27210c8-d99b-4394-94d7-3dc1337a4ac2 admin admin] PATH_INFO: `/v3/projects/ac4c34d742f64d17b81d6a6171e66e84` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:49:37.092719 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27210c8-d99b-4394-94d7-3dc1337a4ac2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac4c34d742f64d17b81d6a6171e66e84)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:49:37.095081 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27210c8-d99b-4394-94d7-3dc1337a4ac2 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:49:37.144969 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-b27210c8-d99b-4394-94d7-3dc1337a4ac2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac4c34d742f64d17b81d6a6171e66e84', 'request_id': 'req-b27210c8-d99b-4394-94d7-3dc1337a4ac2'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:49:37.153232 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-b27210c8-d99b-4394-94d7-3dc1337a4ac2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac4c34d742f64d17b81d6a6171e66e84', 'request_id': 'req-b27210c8-d99b-4394-94d7-3dc1337a4ac2'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:49:37.159591 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-b27210c8-d99b-4394-94d7-3dc1337a4ac2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac4c34d742f64d17b81d6a6171e66e84', 'request_id': 'req-b27210c8-d99b-4394-94d7-3dc1337a4ac2'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:49:37.171228 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-b27210c8-d99b-4394-94d7-3dc1337a4ac2 admin admin] The token cache is being invalidate because project ac4c34d742f64d17b81d6a6171e66e84 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80357) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:49:37.171520 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-b27210c8-d99b-4394-94d7-3dc1337a4ac2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:49:37.172162 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-b27210c8-d99b-4394-94d7-3dc1337a4ac2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:49:37.173768 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1140/2297] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jun 16 17:49:37 2022] DELETE /identity/v3/projects/ac4c34d742f64d17b81d6a6171e66e84 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:49:37.178274 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6273d30-f45c-43b1-994d-e7bc4fa210ee admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:49:37.179693 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-d6273d30-f45c-43b1-994d-e7bc4fa210ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:49:37.193783 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6273d30-f45c-43b1-994d-e7bc4fa210ee admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:49:37.242970 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:49:37.254303 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3a8e23-58cc-4210-9c0b-2165567ce068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:49:37.255168 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3a8e23-58cc-4210-9c0b-2165567ce068 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:49:37.255450 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3a8e23-58cc-4210-9c0b-2165567ce068 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:49:37.255726 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3a8e23-58cc-4210-9c0b-2165567ce068 admin admin] PATH_INFO: `/v3/users/016d0bc441174f408ef50e5eced2e692` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:49:37.256937 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3a8e23-58cc-4210-9c0b-2165567ce068 admin admin] RBAC: Authorizing `identity:delete_user(user_id=016d0bc441174f408ef50e5eced2e692)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:49:37.258853 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3a8e23-58cc-4210-9c0b-2165567ce068 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:49:37.306663 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-1e3a8e23-58cc-4210-9c0b-2165567ce068 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '016d0bc441174f408ef50e5eced2e692'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:49:37.322386 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-1e3a8e23-58cc-4210-9c0b-2165567ce068 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '016d0bc441174f408ef50e5eced2e692'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:49:37.323199 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-1e3a8e23-58cc-4210-9c0b-2165567ce068 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '016d0bc441174f408ef50e5eced2e692'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:49:37.331880 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-1e3a8e23-58cc-4210-9c0b-2165567ce068 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '016d0bc441174f408ef50e5eced2e692'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:49:37.339573 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-1e3a8e23-58cc-4210-9c0b-2165567ce068 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '016d0bc441174f408ef50e5eced2e692'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:49:37.345486 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1158/2298] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jun 16 17:49:37 2022] DELETE /identity/v3/users/016d0bc441174f408ef50e5eced2e692 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:49:37.528588 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27210c8-d99b-4394-94d7-3dc1337a4ac2 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:49:37.541132 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-b27210c8-d99b-4394-94d7-3dc1337a4ac2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:49:37.558487 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27210c8-d99b-4394-94d7-3dc1337a4ac2 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:49:37.563685 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:49:37.625214 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b0a6ef-fa3d-415f-b027-7d605e31cdcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:49:37.626188 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b0a6ef-fa3d-415f-b027-7d605e31cdcf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:49:37.626535 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b0a6ef-fa3d-415f-b027-7d605e31cdcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:49:37.626878 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b0a6ef-fa3d-415f-b027-7d605e31cdcf admin admin] PATH_INFO: `/v3/projects/929cd3d64af3439d9cd63858fcb8fc02` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:49:37.628681 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b0a6ef-fa3d-415f-b027-7d605e31cdcf admin admin] RBAC: Authorizing `identity:delete_project(project_id=929cd3d64af3439d9cd63858fcb8fc02)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:49:37.630607 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b0a6ef-fa3d-415f-b027-7d605e31cdcf admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:49:37.692101 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-e6b0a6ef-fa3d-415f-b027-7d605e31cdcf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '929cd3d64af3439d9cd63858fcb8fc02', 'request_id': 'req-e6b0a6ef-fa3d-415f-b027-7d605e31cdcf'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:49:37.702855 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-e6b0a6ef-fa3d-415f-b027-7d605e31cdcf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '929cd3d64af3439d9cd63858fcb8fc02', 'request_id': 'req-e6b0a6ef-fa3d-415f-b027-7d605e31cdcf'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:49:37.710349 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-e6b0a6ef-fa3d-415f-b027-7d605e31cdcf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '929cd3d64af3439d9cd63858fcb8fc02', 'request_id': 'req-e6b0a6ef-fa3d-415f-b027-7d605e31cdcf'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:49:37.723347 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-e6b0a6ef-fa3d-415f-b027-7d605e31cdcf admin admin] The token cache is being invalidate because project 929cd3d64af3439d9cd63858fcb8fc02 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80357) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:49:37.723636 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-e6b0a6ef-fa3d-415f-b027-7d605e31cdcf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:49:37.724404 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-e6b0a6ef-fa3d-415f-b027-7d605e31cdcf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:49:37.725965 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1141/2299] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jun 16 17:49:37 2022] DELETE /identity/v3/projects/929cd3d64af3439d9cd63858fcb8fc02 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:49:44.574315 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3a8e23-58cc-4210-9c0b-2165567ce068 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:49:44.576058 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-1e3a8e23-58cc-4210-9c0b-2165567ce068 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:49:44.672605 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3a8e23-58cc-4210-9c0b-2165567ce068 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:49:44.752602 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ea164c-906a-44f9-b637-58facee7ff6e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f86c16179a70474a817be9fda33ece9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:49:44.754008 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ea164c-906a-44f9-b637-58facee7ff6e service nova] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:49:44.754459 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ea164c-906a-44f9-b637-58facee7ff6e service nova] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:49:44.754735 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ea164c-906a-44f9-b637-58facee7ff6e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:49:44.758009 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-b4ea164c-906a-44f9-b637-58facee7ff6e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:49:44.765142 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:49:44.838067 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ea164c-906a-44f9-b637-58facee7ff6e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:49:44.839716 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ea164c-906a-44f9-b637-58facee7ff6e service nova] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:49:44.850397 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1159/2300] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:49:44 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:49:45.522519 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b0a6ef-fa3d-415f-b027-7d605e31cdcf admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:49:45.525027 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-e6b0a6ef-fa3d-415f-b027-7d605e31cdcf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:49:45.558602 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b0a6ef-fa3d-415f-b027-7d605e31cdcf admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:49:45.565828 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2527e970-0333-4c5e-9bbd-ec720f6b3e8b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f86c16179a70474a817be9fda33ece9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:49:45.566795 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2527e970-0333-4c5e-9bbd-ec720f6b3e8b service nova] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:49:45.567314 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2527e970-0333-4c5e-9bbd-ec720f6b3e8b service nova] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:49:45.567825 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2527e970-0333-4c5e-9bbd-ec720f6b3e8b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:49:45.569956 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-2527e970-0333-4c5e-9bbd-ec720f6b3e8b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:49:45.628205 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2527e970-0333-4c5e-9bbd-ec720f6b3e8b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:49:45.630232 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2527e970-0333-4c5e-9bbd-ec720f6b3e8b service nova] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:49:45.654938 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1142/2301] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:49:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 17:49:48.923047 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396e5357-39c6-498c-97af-c5d2d2c0af0a None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:49:48.923539 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396e5357-39c6-498c-97af-c5d2d2c0af0a None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:49:48.923639 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396e5357-39c6-498c-97af-c5d2d2c0af0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:49:48.942252 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-396e5357-39c6-498c-97af-c5d2d2c0af0a None None] MFA Rules not processed for user `8fff53cb0d084d9284825488f6843ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:49:48.943430 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-396e5357-39c6-498c-97af-c5d2d2c0af0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:49:48.997351 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1160/2302] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:49:48 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:49:49.338664 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2527e970-0333-4c5e-9bbd-ec720f6b3e8b service nova] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:49:49.340104 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-2527e970-0333-4c5e-9bbd-ec720f6b3e8b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:49:49.396656 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2527e970-0333-4c5e-9bbd-ec720f6b3e8b service nova] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:49:49.400265 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2476424-9cde-4b66-a56e-086870917d00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:49:49.400946 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2476424-9cde-4b66-a56e-086870917d00 service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:49:49.401237 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2476424-9cde-4b66-a56e-086870917d00 service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:49:49.401516 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2476424-9cde-4b66-a56e-086870917d00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:49:49.403258 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-c2476424-9cde-4b66-a56e-086870917d00 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:49:49.413484 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2476424-9cde-4b66-a56e-086870917d00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:49:49.414895 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2476424-9cde-4b66-a56e-086870917d00 service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:49:49.423074 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1143/2303] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:49:49 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:49:49.757988 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396e5357-39c6-498c-97af-c5d2d2c0af0a None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:49:49.759727 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-396e5357-39c6-498c-97af-c5d2d2c0af0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:49:49.836077 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396e5357-39c6-498c-97af-c5d2d2c0af0a None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:49:49.841091 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991e4ee3-cc1a-41af-afd6-19aad0b4bf51 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0570d8d9508d4546ae897276f7d181d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:49:49.841741 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991e4ee3-cc1a-41af-afd6-19aad0b4bf51 service placement] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:49:49.842022 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991e4ee3-cc1a-41af-afd6-19aad0b4bf51 service placement] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:49:49.842412 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991e4ee3-cc1a-41af-afd6-19aad0b4bf51 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:49:49.844398 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-991e4ee3-cc1a-41af-afd6-19aad0b4bf51 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:49:49.857291 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991e4ee3-cc1a-41af-afd6-19aad0b4bf51 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:49:49.858884 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991e4ee3-cc1a-41af-afd6-19aad0b4bf51 service placement] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:49:49.872911 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1161/2304] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:49:49 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:49:54.063058 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2476424-9cde-4b66-a56e-086870917d00 service neutron] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:49:54.072456 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2476424-9cde-4b66-a56e-086870917d00 service neutron] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:49:54.076208 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2efac18-793e-4ef4-a407-f0f94310ddef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:49:54.076886 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2efac18-793e-4ef4-a407-f0f94310ddef service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:49:54.077176 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2efac18-793e-4ef4-a407-f0f94310ddef service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:49:54.077453 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2efac18-793e-4ef4-a407-f0f94310ddef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:49:54.079347 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-d2efac18-793e-4ef4-a407-f0f94310ddef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:49:54.087754 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991e4ee3-cc1a-41af-afd6-19aad0b4bf51 service placement] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:49:54.090686 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2efac18-793e-4ef4-a407-f0f94310ddef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:49:54.092178 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2efac18-793e-4ef4-a407-f0f94310ddef service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:49:54.098059 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991e4ee3-cc1a-41af-afd6-19aad0b4bf51 service placement] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:49:54.103611 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa03a193-ecb1-4deb-85e9-896636f3ed58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:49:54.104455 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa03a193-ecb1-4deb-85e9-896636f3ed58 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:49:54.104855 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa03a193-ecb1-4deb-85e9-896636f3ed58 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:49:54.105227 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa03a193-ecb1-4deb-85e9-896636f3ed58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:49:54.107898 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1144/2305] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:49:54 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:49:54.116104 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa03a193-ecb1-4deb-85e9-896636f3ed58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:49:54.118528 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa03a193-ecb1-4deb-85e9-896636f3ed58 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:49:54.129361 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1162/2306] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:49:54 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 17:50:00.311330 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2efac18-793e-4ef4-a407-f0f94310ddef service neutron] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:00.312782 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-d2efac18-793e-4ef4-a407-f0f94310ddef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:50:00.323800 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2efac18-793e-4ef4-a407-f0f94310ddef service neutron] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:00.327528 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc36e5d-0988-458e-874c-07ae9406aec8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:00.328163 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc36e5d-0988-458e-874c-07ae9406aec8 service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:00.328489 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc36e5d-0988-458e-874c-07ae9406aec8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:00.328769 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc36e5d-0988-458e-874c-07ae9406aec8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:00.330467 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-ddc36e5d-0988-458e-874c-07ae9406aec8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:50:00.340432 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:50:00.395473 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc36e5d-0988-458e-874c-07ae9406aec8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:00.396874 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc36e5d-0988-458e-874c-07ae9406aec8 service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:00.406520 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1145/2307] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:50:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 17:50:03.642781 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa03a193-ecb1-4deb-85e9-896636f3ed58 service neutron] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:03.645007 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-aa03a193-ecb1-4deb-85e9-896636f3ed58 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:50:03.660818 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa03a193-ecb1-4deb-85e9-896636f3ed58 service neutron] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:03.670015 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14e04bd-bbaa-4c3f-9a7c-2f56252b7af9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:03.670894 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14e04bd-bbaa-4c3f-9a7c-2f56252b7af9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:03.671181 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14e04bd-bbaa-4c3f-9a7c-2f56252b7af9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:03.671462 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14e04bd-bbaa-4c3f-9a7c-2f56252b7af9 admin admin] PATH_INFO: `/v3/users/268c5947a265492bb7a8f181f9f5c504` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:03.672789 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14e04bd-bbaa-4c3f-9a7c-2f56252b7af9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=268c5947a265492bb7a8f181f9f5c504)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:03.674718 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14e04bd-bbaa-4c3f-9a7c-2f56252b7af9 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:03.698627 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:50:03.729275 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-d14e04bd-bbaa-4c3f-9a7c-2f56252b7af9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '268c5947a265492bb7a8f181f9f5c504'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:03.741554 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-d14e04bd-bbaa-4c3f-9a7c-2f56252b7af9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '268c5947a265492bb7a8f181f9f5c504'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:03.742244 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-d14e04bd-bbaa-4c3f-9a7c-2f56252b7af9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '268c5947a265492bb7a8f181f9f5c504'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:03.749052 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-d14e04bd-bbaa-4c3f-9a7c-2f56252b7af9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '268c5947a265492bb7a8f181f9f5c504'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:03.756573 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-d14e04bd-bbaa-4c3f-9a7c-2f56252b7af9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '268c5947a265492bb7a8f181f9f5c504'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:03.762725 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1163/2308] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jun 16 17:50:03 2022] DELETE /identity/v3/users/268c5947a265492bb7a8f181f9f5c504 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:50:03.944913 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc36e5d-0988-458e-874c-07ae9406aec8 service neutron] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:03.963483 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc36e5d-0988-458e-874c-07ae9406aec8 service neutron] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:04.032763 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c07e49-e9ab-4756-8477-d17c8b765362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:04.033659 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c07e49-e9ab-4756-8477-d17c8b765362 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:04.035137 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c07e49-e9ab-4756-8477-d17c8b765362 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:04.035416 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c07e49-e9ab-4756-8477-d17c8b765362 admin admin] PATH_INFO: `/v3/projects/feb571d68c6c4a1ab9e7b175f5a6e0b3` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:04.036729 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c07e49-e9ab-4756-8477-d17c8b765362 admin admin] RBAC: Authorizing `identity:delete_project(project_id=feb571d68c6c4a1ab9e7b175f5a6e0b3)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:04.038713 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c07e49-e9ab-4756-8477-d17c8b765362 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:04.089146 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-24c07e49-e9ab-4756-8477-d17c8b765362 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'feb571d68c6c4a1ab9e7b175f5a6e0b3', 'request_id': 'req-24c07e49-e9ab-4756-8477-d17c8b765362'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:04.098454 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-24c07e49-e9ab-4756-8477-d17c8b765362 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'feb571d68c6c4a1ab9e7b175f5a6e0b3', 'request_id': 'req-24c07e49-e9ab-4756-8477-d17c8b765362'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:04.104074 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-24c07e49-e9ab-4756-8477-d17c8b765362 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'feb571d68c6c4a1ab9e7b175f5a6e0b3', 'request_id': 'req-24c07e49-e9ab-4756-8477-d17c8b765362'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:04.116438 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-24c07e49-e9ab-4756-8477-d17c8b765362 admin admin] The token cache is being invalidate because project feb571d68c6c4a1ab9e7b175f5a6e0b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80357) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:50:04.116722 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-24c07e49-e9ab-4756-8477-d17c8b765362 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:04.117367 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-24c07e49-e9ab-4756-8477-d17c8b765362 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:04.118918 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1146/2309] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jun 16 17:50:03 2022] DELETE /identity/v3/projects/feb571d68c6c4a1ab9e7b175f5a6e0b3 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:50:04.123836 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6424cbd1-4d0d-44ad-86ea-27814a6e507b None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:04.124134 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6424cbd1-4d0d-44ad-86ea-27814a6e507b None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:04.124449 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6424cbd1-4d0d-44ad-86ea-27814a6e507b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:04.144862 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-6424cbd1-4d0d-44ad-86ea-27814a6e507b None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:50:04.146281 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-6424cbd1-4d0d-44ad-86ea-27814a6e507b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:50:04.207779 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c07e49-e9ab-4756-8477-d17c8b765362 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:04.209238 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-24c07e49-e9ab-4756-8477-d17c8b765362 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:50:04.226226 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1164/2310] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:50:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:50:04.235468 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c1237e-e3d0-472a-b967-8eded075e5ec None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:04.235723 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c1237e-e3d0-472a-b967-8eded075e5ec None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:04.236146 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c1237e-e3d0-472a-b967-8eded075e5ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:04.259275 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-01c1237e-e3d0-472a-b967-8eded075e5ec None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:50:04.260548 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-01c1237e-e3d0-472a-b967-8eded075e5ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:50:04.273093 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1165/2311] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:50:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:50:04.277309 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c1237e-e3d0-472a-b967-8eded075e5ec None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:04.294430 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:50:04.295004 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c1237e-e3d0-472a-b967-8eded075e5ec None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:04.308526 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a02d27a-0883-45f5-bfc9-5691c7ae7f60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:04.309757 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a02d27a-0883-45f5-bfc9-5691c7ae7f60 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:04.311370 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a02d27a-0883-45f5-bfc9-5691c7ae7f60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:04.311724 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a02d27a-0883-45f5-bfc9-5691c7ae7f60 admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:04.312911 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a02d27a-0883-45f5-bfc9-5691c7ae7f60 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:50:04.313363 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a02d27a-0883-45f5-bfc9-5691c7ae7f60 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:04.315187 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a02d27a-0883-45f5-bfc9-5691c7ae7f60 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:04.332009 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1166/2312] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jun 16 17:50:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:50:04.336256 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a02d27a-0883-45f5-bfc9-5691c7ae7f60 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:04.348196 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a02d27a-0883-45f5-bfc9-5691c7ae7f60 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:04.348751 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c07e49-e9ab-4756-8477-d17c8b765362 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:04.354028 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c32125-4f49-4d96-976d-ec21cd330de1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:04.354957 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c32125-4f49-4d96-976d-ec21cd330de1 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:04.355090 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79213cc-de21-4987-ac55-3919a986447a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f86c16179a70474a817be9fda33ece9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:04.355540 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c32125-4f49-4d96-976d-ec21cd330de1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:04.355672 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79213cc-de21-4987-ac55-3919a986447a service nova] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:04.355883 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79213cc-de21-4987-ac55-3919a986447a service nova] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:04.356163 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79213cc-de21-4987-ac55-3919a986447a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:04.356570 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c32125-4f49-4d96-976d-ec21cd330de1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:04.357437 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c32125-4f49-4d96-976d-ec21cd330de1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:04.357954 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-d79213cc-de21-4987-ac55-3919a986447a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:50:04.359696 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c32125-4f49-4d96-976d-ec21cd330de1 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:04.371731 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79213cc-de21-4987-ac55-3919a986447a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:04.373402 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79213cc-de21-4987-ac55-3919a986447a service nova] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:04.387497 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1147/2313] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:50:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:50:04.408262 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1167/2314] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jun 16 17:50:04 2022] POST /identity/v3/projects => generated 375 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:50:04.410161 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79213cc-de21-4987-ac55-3919a986447a service nova] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:04.421864 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79213cc-de21-4987-ac55-3919a986447a service nova] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:04.492244 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de4924e-3f35-4ede-90b2-bea2889b511f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:04.493147 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de4924e-3f35-4ede-90b2-bea2889b511f admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:04.493446 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de4924e-3f35-4ede-90b2-bea2889b511f admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:04.493848 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de4924e-3f35-4ede-90b2-bea2889b511f admin admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:04.494648 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de4924e-3f35-4ede-90b2-bea2889b511f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:04.496942 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de4924e-3f35-4ede-90b2-bea2889b511f admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:04.523864 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1148/2315] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jun 16 17:50:04 2022] POST /identity/v3/users => generated 288 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:50:04.528023 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c32125-4f49-4d96-976d-ec21cd330de1 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:04.543219 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c32125-4f49-4d96-976d-ec21cd330de1 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:04.548304 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27bde8c-4c87-4dda-9eee-9d77cca5305d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:04.549010 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27bde8c-4c87-4dda-9eee-9d77cca5305d admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:04.549294 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27bde8c-4c87-4dda-9eee-9d77cca5305d admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:04.550387 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27bde8c-4c87-4dda-9eee-9d77cca5305d admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:04.551813 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27bde8c-4c87-4dda-9eee-9d77cca5305d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:04.553703 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27bde8c-4c87-4dda-9eee-9d77cca5305d admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:04.564413 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1168/2316] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:50:04 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:50:04.568683 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de4924e-3f35-4ede-90b2-bea2889b511f admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:04.579457 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de4924e-3f35-4ede-90b2-bea2889b511f admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:04.585505 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1246ff10-81f7-46a5-ba9c-47ecbbf5d4f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:04.586177 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1246ff10-81f7-46a5-ba9c-47ecbbf5d4f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:04.586461 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1246ff10-81f7-46a5-ba9c-47ecbbf5d4f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:04.586736 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1246ff10-81f7-46a5-ba9c-47ecbbf5d4f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:04.587465 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1246ff10-81f7-46a5-ba9c-47ecbbf5d4f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:04.589167 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1246ff10-81f7-46a5-ba9c-47ecbbf5d4f9 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:04.601268 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1149/2317] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:50:04 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:50:04.605592 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27bde8c-4c87-4dda-9eee-9d77cca5305d admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:04.617957 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27bde8c-4c87-4dda-9eee-9d77cca5305d admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:04.622864 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efa5dc1-1082-4fb8-960e-7423e59d8741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:04.623759 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efa5dc1-1082-4fb8-960e-7423e59d8741 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:04.624119 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efa5dc1-1082-4fb8-960e-7423e59d8741 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:04.625512 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efa5dc1-1082-4fb8-960e-7423e59d8741 admin admin] PATH_INFO: `/v3/projects/818b250f052347ea90f850675c2976c3/users/efc81237995e45cface80f5f9bd19c3e/roles/0be2075946f54c8485f1c275f746478a` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:04.646742 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efa5dc1-1082-4fb8-960e-7423e59d8741 admin admin] RBAC: Authorizing `identity:create_grant(project_id=818b250f052347ea90f850675c2976c3, user_id=efc81237995e45cface80f5f9bd19c3e, role_id=0be2075946f54c8485f1c275f746478a)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:04.652357 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efa5dc1-1082-4fb8-960e-7423e59d8741 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:04.668423 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1169/2318] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:50:04 2022] PUT /identity/v3/projects/818b250f052347ea90f850675c2976c3/users/efc81237995e45cface80f5f9bd19c3e/roles/0be2075946f54c8485f1c275f746478a => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:50:04.677648 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b485e07f-33b3-473a-b802-e53f01869cb8 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:04.678159 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b485e07f-33b3-473a-b802-e53f01869cb8 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:04.678704 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b485e07f-33b3-473a-b802-e53f01869cb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:04.710384 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-b485e07f-33b3-473a-b802-e53f01869cb8 None None] MFA Rules not processed for user `efc81237995e45cface80f5f9bd19c3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:50:04.712087 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-b485e07f-33b3-473a-b802-e53f01869cb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:50:04.799131 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1150/2319] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:50:04 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:50:04.810049 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efa5dc1-1082-4fb8-960e-7423e59d8741 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:04.828201 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=91,keystone:DELETE=6,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:50:04.883055 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efa5dc1-1082-4fb8-960e-7423e59d8741 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:04.889490 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208a08f7-afe6-4012-8b94-33afc8ffece5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f86c16179a70474a817be9fda33ece9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:04.890582 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208a08f7-afe6-4012-8b94-33afc8ffece5 service nova] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:04.890582 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208a08f7-afe6-4012-8b94-33afc8ffece5 service nova] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:04.890842 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208a08f7-afe6-4012-8b94-33afc8ffece5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:04.899515 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208a08f7-afe6-4012-8b94-33afc8ffece5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:04.901128 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208a08f7-afe6-4012-8b94-33afc8ffece5 service nova] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:04.910913 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1170/2320] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:50:04 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 17:50:05.138787 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b485e07f-33b3-473a-b802-e53f01869cb8 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:05.172785 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:50:05.203888 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b485e07f-33b3-473a-b802-e53f01869cb8 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:05.209572 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c330363f-3540-4532-a792-cdff8cf59aa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:05.210433 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c330363f-3540-4532-a792-cdff8cf59aa4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:05.210716 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c330363f-3540-4532-a792-cdff8cf59aa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:05.210993 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c330363f-3540-4532-a792-cdff8cf59aa4 admin admin] PATH_INFO: `/v3/users/efc81237995e45cface80f5f9bd19c3e` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:05.212171 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c330363f-3540-4532-a792-cdff8cf59aa4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=efc81237995e45cface80f5f9bd19c3e)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:05.214132 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c330363f-3540-4532-a792-cdff8cf59aa4 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:05.262103 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-c330363f-3540-4532-a792-cdff8cf59aa4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'efc81237995e45cface80f5f9bd19c3e'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:05.262751 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-c330363f-3540-4532-a792-cdff8cf59aa4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'efc81237995e45cface80f5f9bd19c3e'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:05.270947 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-c330363f-3540-4532-a792-cdff8cf59aa4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'efc81237995e45cface80f5f9bd19c3e'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:05.271568 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-c330363f-3540-4532-a792-cdff8cf59aa4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'efc81237995e45cface80f5f9bd19c3e'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:05.284093 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-c330363f-3540-4532-a792-cdff8cf59aa4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'efc81237995e45cface80f5f9bd19c3e'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:05.298004 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1151/2321] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jun 16 17:50:05 2022] DELETE /identity/v3/users/efc81237995e45cface80f5f9bd19c3e => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:50:05.302620 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164d7036-0d6b-4e83-bfb0-a92b8b3aad7b None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:05.302849 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164d7036-0d6b-4e83-bfb0-a92b8b3aad7b None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:05.303128 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164d7036-0d6b-4e83-bfb0-a92b8b3aad7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:05.320917 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-164d7036-0d6b-4e83-bfb0-a92b8b3aad7b None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:50:05.322099 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-164d7036-0d6b-4e83-bfb0-a92b8b3aad7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:50:05.387063 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1171/2322] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:50:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:50:05.396931 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c330363f-3540-4532-a792-cdff8cf59aa4 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:05.399199 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-c330363f-3540-4532-a792-cdff8cf59aa4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:50:05.460643 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c330363f-3540-4532-a792-cdff8cf59aa4 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:05.464363 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42adb5f2-b714-4f78-a8ba-6fb6a0666a0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:05.465038 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42adb5f2-b714-4f78-a8ba-6fb6a0666a0a service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:05.465322 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42adb5f2-b714-4f78-a8ba-6fb6a0666a0a service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:05.465598 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42adb5f2-b714-4f78-a8ba-6fb6a0666a0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:05.475531 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42adb5f2-b714-4f78-a8ba-6fb6a0666a0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:05.477043 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42adb5f2-b714-4f78-a8ba-6fb6a0666a0a service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:05.488778 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1152/2323] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:50:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:50:05.761980 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164d7036-0d6b-4e83-bfb0-a92b8b3aad7b None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:05.763417 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-164d7036-0d6b-4e83-bfb0-a92b8b3aad7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:50:05.777593 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164d7036-0d6b-4e83-bfb0-a92b8b3aad7b None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:05.782391 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039ab0b7-582d-4c99-a22c-fffca7089afa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:05.783399 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039ab0b7-582d-4c99-a22c-fffca7089afa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:05.784607 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039ab0b7-582d-4c99-a22c-fffca7089afa admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:05.784607 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039ab0b7-582d-4c99-a22c-fffca7089afa admin admin] PATH_INFO: `/v3/projects/818b250f052347ea90f850675c2976c3` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:05.786526 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039ab0b7-582d-4c99-a22c-fffca7089afa admin admin] RBAC: Authorizing `identity:delete_project(project_id=818b250f052347ea90f850675c2976c3)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:05.788424 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039ab0b7-582d-4c99-a22c-fffca7089afa admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:05.835866 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-039ab0b7-582d-4c99-a22c-fffca7089afa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '818b250f052347ea90f850675c2976c3', 'request_id': 'req-039ab0b7-582d-4c99-a22c-fffca7089afa'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:05.844554 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-039ab0b7-582d-4c99-a22c-fffca7089afa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '818b250f052347ea90f850675c2976c3', 'request_id': 'req-039ab0b7-582d-4c99-a22c-fffca7089afa'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:05.856984 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-039ab0b7-582d-4c99-a22c-fffca7089afa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '818b250f052347ea90f850675c2976c3', 'request_id': 'req-039ab0b7-582d-4c99-a22c-fffca7089afa'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:05.869713 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-039ab0b7-582d-4c99-a22c-fffca7089afa admin admin] The token cache is being invalidate because project 818b250f052347ea90f850675c2976c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:50:05.870035 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-039ab0b7-582d-4c99-a22c-fffca7089afa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:05.870671 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-039ab0b7-582d-4c99-a22c-fffca7089afa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:05.872412 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1172/2324] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jun 16 17:50:05 2022] DELETE /identity/v3/projects/818b250f052347ea90f850675c2976c3 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:50:05.877185 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558fd55b-6c02-45ab-aff2-2fcf2eb4b4f4 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:05.877517 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558fd55b-6c02-45ab-aff2-2fcf2eb4b4f4 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:05.877788 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558fd55b-6c02-45ab-aff2-2fcf2eb4b4f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:05.901882 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-558fd55b-6c02-45ab-aff2-2fcf2eb4b4f4 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:50:05.903119 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-558fd55b-6c02-45ab-aff2-2fcf2eb4b4f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:50:05.960848 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:50:05.980596 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1153/2325] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:50:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:50:05.988220 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3927ea5-4841-49df-b8b2-5dca7bd65552 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:05.988667 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3927ea5-4841-49df-b8b2-5dca7bd65552 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:05.988984 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3927ea5-4841-49df-b8b2-5dca7bd65552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:06.010396 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-c3927ea5-4841-49df-b8b2-5dca7bd65552 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:50:06.012267 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-c3927ea5-4841-49df-b8b2-5dca7bd65552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:50:06.028511 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1173/2326] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:50:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:50:06.033111 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558fd55b-6c02-45ab-aff2-2fcf2eb4b4f4 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:06.046458 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558fd55b-6c02-45ab-aff2-2fcf2eb4b4f4 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:06.061699 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85df5c8-a5cd-44eb-ad16-3d73244a5b22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:06.062397 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85df5c8-a5cd-44eb-ad16-3d73244a5b22 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:06.062685 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85df5c8-a5cd-44eb-ad16-3d73244a5b22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:06.062964 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85df5c8-a5cd-44eb-ad16-3d73244a5b22 admin admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:06.063704 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85df5c8-a5cd-44eb-ad16-3d73244a5b22 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:50:06.064013 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85df5c8-a5cd-44eb-ad16-3d73244a5b22 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:06.066127 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85df5c8-a5cd-44eb-ad16-3d73244a5b22 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:06.079424 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1154/2327] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jun 16 17:50:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:50:06.084519 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3927ea5-4841-49df-b8b2-5dca7bd65552 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:06.098785 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3927ea5-4841-49df-b8b2-5dca7bd65552 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:06.104340 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa83b4b2-a1c4-4edc-b67b-fd40c6704841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:06.105235 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa83b4b2-a1c4-4edc-b67b-fd40c6704841 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:06.105700 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa83b4b2-a1c4-4edc-b67b-fd40c6704841 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:06.106142 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa83b4b2-a1c4-4edc-b67b-fd40c6704841 admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:06.107046 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa83b4b2-a1c4-4edc-b67b-fd40c6704841 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:06.109558 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa83b4b2-a1c4-4edc-b67b-fd40c6704841 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:06.129859 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:50:06.144843 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1174/2328] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jun 16 17:50:06 2022] POST /identity/v3/projects => generated 369 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:50:06.148979 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85df5c8-a5cd-44eb-ad16-3d73244a5b22 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:06.159390 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85df5c8-a5cd-44eb-ad16-3d73244a5b22 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:06.215133 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adccb7c-2b0d-4ac1-a6b4-e87cecc2c1fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:06.215801 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adccb7c-2b0d-4ac1-a6b4-e87cecc2c1fe admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:06.216088 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adccb7c-2b0d-4ac1-a6b4-e87cecc2c1fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:06.216366 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adccb7c-2b0d-4ac1-a6b4-e87cecc2c1fe admin admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:06.217078 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adccb7c-2b0d-4ac1-a6b4-e87cecc2c1fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:06.219001 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adccb7c-2b0d-4ac1-a6b4-e87cecc2c1fe admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:06.238823 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1155/2329] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jun 16 17:50:06 2022] POST /identity/v3/users => generated 285 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:50:06.242807 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa83b4b2-a1c4-4edc-b67b-fd40c6704841 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:06.253929 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa83b4b2-a1c4-4edc-b67b-fd40c6704841 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:06.258506 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db25095-efa8-444a-88ac-613646167807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:06.259157 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db25095-efa8-444a-88ac-613646167807 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:06.259438 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db25095-efa8-444a-88ac-613646167807 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:06.259712 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db25095-efa8-444a-88ac-613646167807 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:06.260457 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db25095-efa8-444a-88ac-613646167807 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:06.262302 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db25095-efa8-444a-88ac-613646167807 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:06.272659 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1175/2330] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:50:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:50:06.276651 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adccb7c-2b0d-4ac1-a6b4-e87cecc2c1fe admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:06.286803 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adccb7c-2b0d-4ac1-a6b4-e87cecc2c1fe admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:06.291378 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4794cc04-ec87-4ba2-a8ac-df550185a494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:06.292058 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4794cc04-ec87-4ba2-a8ac-df550185a494 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:06.292330 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4794cc04-ec87-4ba2-a8ac-df550185a494 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:06.292679 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4794cc04-ec87-4ba2-a8ac-df550185a494 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:06.293356 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4794cc04-ec87-4ba2-a8ac-df550185a494 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:06.294957 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4794cc04-ec87-4ba2-a8ac-df550185a494 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:06.306627 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1156/2331] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:50:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 17:50:06.311138 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db25095-efa8-444a-88ac-613646167807 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:06.322403 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db25095-efa8-444a-88ac-613646167807 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:06.327386 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d6d892-bbd9-42ff-b967-169bbe5b6d71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:06.328389 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d6d892-bbd9-42ff-b967-169bbe5b6d71 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:06.328749 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d6d892-bbd9-42ff-b967-169bbe5b6d71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:06.329096 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d6d892-bbd9-42ff-b967-169bbe5b6d71 admin admin] PATH_INFO: `/v3/projects/2411977f39484513adcaf653cd25ab58/users/0b50a0e55730403cb3246424988d3819/roles/0be2075946f54c8485f1c275f746478a` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:06.344865 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d6d892-bbd9-42ff-b967-169bbe5b6d71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2411977f39484513adcaf653cd25ab58, user_id=0b50a0e55730403cb3246424988d3819, role_id=0be2075946f54c8485f1c275f746478a)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:06.348114 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d6d892-bbd9-42ff-b967-169bbe5b6d71 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:06.362469 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1176/2332] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:50:06 2022] PUT /identity/v3/projects/2411977f39484513adcaf653cd25ab58/users/0b50a0e55730403cb3246424988d3819/roles/0be2075946f54c8485f1c275f746478a => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:50:06.372662 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c65e63-ed0d-4eff-9ec6-123517f8df5c None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:06.372947 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c65e63-ed0d-4eff-9ec6-123517f8df5c None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:06.373245 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c65e63-ed0d-4eff-9ec6-123517f8df5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:06.401803 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-30c65e63-ed0d-4eff-9ec6-123517f8df5c None None] MFA Rules not processed for user `0b50a0e55730403cb3246424988d3819`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:50:06.403018 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-30c65e63-ed0d-4eff-9ec6-123517f8df5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:50:06.496788 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1157/2333] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:50:06 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:50:06.501210 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d6d892-bbd9-42ff-b967-169bbe5b6d71 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:06.568184 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d6d892-bbd9-42ff-b967-169bbe5b6d71 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:06.573339 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534fc271-3ac9-4929-bf95-2bf28ebb8c4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:06.574085 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534fc271-3ac9-4929-bf95-2bf28ebb8c4d admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:06.574445 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534fc271-3ac9-4929-bf95-2bf28ebb8c4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:06.574809 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534fc271-3ac9-4929-bf95-2bf28ebb8c4d admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:06.575570 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534fc271-3ac9-4929-bf95-2bf28ebb8c4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:06.577509 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534fc271-3ac9-4929-bf95-2bf28ebb8c4d admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:06.607930 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1177/2334] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jun 16 17:50:06 2022] POST /identity/v3/projects => generated 373 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:50:06.609375 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c65e63-ed0d-4eff-9ec6-123517f8df5c None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:06.621961 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c65e63-ed0d-4eff-9ec6-123517f8df5c None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:06.673290 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:50:06.687089 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75e06d5-5b87-44c5-9362-cd6d8a13fe78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:06.687801 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75e06d5-5b87-44c5-9362-cd6d8a13fe78 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:06.688090 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75e06d5-5b87-44c5-9362-cd6d8a13fe78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:06.688397 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75e06d5-5b87-44c5-9362-cd6d8a13fe78 admin admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:06.689126 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75e06d5-5b87-44c5-9362-cd6d8a13fe78 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:06.691131 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75e06d5-5b87-44c5-9362-cd6d8a13fe78 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:06.711365 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1158/2335] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jun 16 17:50:06 2022] POST /identity/v3/users => generated 293 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:50:06.715362 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534fc271-3ac9-4929-bf95-2bf28ebb8c4d admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:06.726369 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534fc271-3ac9-4929-bf95-2bf28ebb8c4d admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:06.731118 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1daf4c-8e0d-4b10-845b-cef6af35b333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:06.731806 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1daf4c-8e0d-4b10-845b-cef6af35b333 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:06.732152 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1daf4c-8e0d-4b10-845b-cef6af35b333 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:06.732473 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1daf4c-8e0d-4b10-845b-cef6af35b333 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:06.733262 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1daf4c-8e0d-4b10-845b-cef6af35b333 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:06.734994 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1daf4c-8e0d-4b10-845b-cef6af35b333 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:06.746537 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1178/2336] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:50:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:50:06.750593 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75e06d5-5b87-44c5-9362-cd6d8a13fe78 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:06.761329 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75e06d5-5b87-44c5-9362-cd6d8a13fe78 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:06.765967 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8763c8fe-8e79-4505-b8e0-bffff8d45a25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:06.766837 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8763c8fe-8e79-4505-b8e0-bffff8d45a25 admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:06.767118 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8763c8fe-8e79-4505-b8e0-bffff8d45a25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:06.767394 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8763c8fe-8e79-4505-b8e0-bffff8d45a25 admin admin] PATH_INFO: `/v3/projects/9ade688548524aa6b7a7f01300416c5e/users/0ea050c26a554c00b2c92ef57caffa7c/roles/3ca2dcbbd29d4a31a3ab78f2299bc239` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:06.783244 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8763c8fe-8e79-4505-b8e0-bffff8d45a25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ade688548524aa6b7a7f01300416c5e, user_id=0ea050c26a554c00b2c92ef57caffa7c, role_id=3ca2dcbbd29d4a31a3ab78f2299bc239)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:06.786920 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8763c8fe-8e79-4505-b8e0-bffff8d45a25 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:06.800778 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1159/2337] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:50:06 2022] PUT /identity/v3/projects/9ade688548524aa6b7a7f01300416c5e/users/0ea050c26a554c00b2c92ef57caffa7c/roles/3ca2dcbbd29d4a31a3ab78f2299bc239 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:50:06.805084 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1daf4c-8e0d-4b10-845b-cef6af35b333 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:06.815328 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1daf4c-8e0d-4b10-845b-cef6af35b333 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:06.872223 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6306319-4aca-48b1-b2e4-112bab8a3806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:06.872848 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6306319-4aca-48b1-b2e4-112bab8a3806 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:06.873129 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6306319-4aca-48b1-b2e4-112bab8a3806 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:06.873404 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6306319-4aca-48b1-b2e4-112bab8a3806 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:06.874127 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6306319-4aca-48b1-b2e4-112bab8a3806 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:06.875719 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6306319-4aca-48b1-b2e4-112bab8a3806 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:06.886480 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1179/2338] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:50:06 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:50:06.890690 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8763c8fe-8e79-4505-b8e0-bffff8d45a25 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:06.902527 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8763c8fe-8e79-4505-b8e0-bffff8d45a25 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:06.907045 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74925980-2793-4e0b-8eb9-15d3bde19f09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:06.908006 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74925980-2793-4e0b-8eb9-15d3bde19f09 admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:06.908287 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74925980-2793-4e0b-8eb9-15d3bde19f09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:06.908597 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74925980-2793-4e0b-8eb9-15d3bde19f09 admin admin] PATH_INFO: `/v3/projects/9ade688548524aa6b7a7f01300416c5e/users/0ea050c26a554c00b2c92ef57caffa7c/roles/3ca2dcbbd29d4a31a3ab78f2299bc239` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:06.911193 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74925980-2793-4e0b-8eb9-15d3bde19f09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ade688548524aa6b7a7f01300416c5e, user_id=0ea050c26a554c00b2c92ef57caffa7c, role_id=3ca2dcbbd29d4a31a3ab78f2299bc239)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:06.913853 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74925980-2793-4e0b-8eb9-15d3bde19f09 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:06.923895 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1160/2339] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:50:06 2022] PUT /identity/v3/projects/9ade688548524aa6b7a7f01300416c5e/users/0ea050c26a554c00b2c92ef57caffa7c/roles/3ca2dcbbd29d4a31a3ab78f2299bc239 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:50:06.935021 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75dcf58-1a9c-4a45-ade9-cb15f4a8da2f None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:06.935311 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75dcf58-1a9c-4a45-ade9-cb15f4a8da2f None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:06.935590 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75dcf58-1a9c-4a45-ade9-cb15f4a8da2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:06.963530 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-a75dcf58-1a9c-4a45-ade9-cb15f4a8da2f None None] MFA Rules not processed for user `0ea050c26a554c00b2c92ef57caffa7c`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:50:06.964780 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-a75dcf58-1a9c-4a45-ade9-cb15f4a8da2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:50:06.983266 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:50:07.033646 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1180/2340] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:50:06 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:50:07.046074 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74925980-2793-4e0b-8eb9-15d3bde19f09 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:07.047832 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-74925980-2793-4e0b-8eb9-15d3bde19f09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:50:07.108627 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74925980-2793-4e0b-8eb9-15d3bde19f09 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:07.112605 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2b8aa7-28f2-41df-ab1c-91b9b1739543 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24d5e2637b7043e08313909d5b06e329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:07.113265 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2b8aa7-28f2-41df-ab1c-91b9b1739543 service cinder] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:07.113553 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2b8aa7-28f2-41df-ab1c-91b9b1739543 service cinder] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:07.113839 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2b8aa7-28f2-41df-ab1c-91b9b1739543 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:07.123486 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2b8aa7-28f2-41df-ab1c-91b9b1739543 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:07.124942 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2b8aa7-28f2-41df-ab1c-91b9b1739543 service cinder] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:07.135004 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1161/2341] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:50:07 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:50:07.222136 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75dcf58-1a9c-4a45-ade9-cb15f4a8da2f None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:07.233879 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75dcf58-1a9c-4a45-ade9-cb15f4a8da2f None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:07.238455 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98694a10-4fba-4430-a435-7d3d501c3205 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24d5e2637b7043e08313909d5b06e329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:07.239243 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98694a10-4fba-4430-a435-7d3d501c3205 service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:07.239619 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98694a10-4fba-4430-a435-7d3d501c3205 service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:07.240105 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98694a10-4fba-4430-a435-7d3d501c3205 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:07.334149 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98694a10-4fba-4430-a435-7d3d501c3205 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:07.335583 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98694a10-4fba-4430-a435-7d3d501c3205 service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:07.345992 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1181/2342] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:50:07 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:50:07.475455 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8106de4c-0884-46a0-abf3-9795933ff399 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:07.476032 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8106de4c-0884-46a0-abf3-9795933ff399 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:07.476032 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8106de4c-0884-46a0-abf3-9795933ff399 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:07.498731 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-8106de4c-0884-46a0-abf3-9795933ff399 None None] MFA Rules not processed for user `71443c30b4554719b5549493bf3cc00c`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:50:07.500330 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-8106de4c-0884-46a0-abf3-9795933ff399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:50:07.576295 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1162/2343] 10.222.0.42 () {58 vars in 1009 bytes} [Thu Jun 16 17:50:07 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:50:07.591133 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98694a10-4fba-4430-a435-7d3d501c3205 service cinder] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:07.592712 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-98694a10-4fba-4430-a435-7d3d501c3205 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:50:07.677254 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98694a10-4fba-4430-a435-7d3d501c3205 service cinder] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:07.682715 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23017399-c735-4c7a-aebc-1d20faf5e7e0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47b7a9cd615c41d6983a0007b475038c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:07.683360 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23017399-c735-4c7a-aebc-1d20faf5e7e0 service swift] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:07.683633 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23017399-c735-4c7a-aebc-1d20faf5e7e0 service swift] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:07.683912 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23017399-c735-4c7a-aebc-1d20faf5e7e0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:07.700102 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23017399-c735-4c7a-aebc-1d20faf5e7e0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:07.701822 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23017399-c735-4c7a-aebc-1d20faf5e7e0 service swift] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:07.712817 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1182/2344] 10.222.0.42 () {60 vars in 1384 bytes} [Thu Jun 16 17:50:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 16 17:50:17.567888 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=73,keystone:INSERT=5 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:50:17.708468 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=80 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:50:23.460215 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383fcfe0-9d02-44c4-bb52-7b497d922f93 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:23.460823 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383fcfe0-9d02-44c4-bb52-7b497d922f93 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:23.460823 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383fcfe0-9d02-44c4-bb52-7b497d922f93 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:23.462194 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1163/2345] 10.222.0.42 () {54 vars in 921 bytes} [Thu Jun 16 17:50:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:50:23.468812 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218e4d55-8d29-4110-819d-fdd711ece3e0 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:23.469240 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218e4d55-8d29-4110-819d-fdd711ece3e0 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:23.469387 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218e4d55-8d29-4110-819d-fdd711ece3e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:23.489650 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-218e4d55-8d29-4110-819d-fdd711ece3e0 None None] MFA Rules not processed for user `f86c16179a70474a817be9fda33ece9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:50:23.490960 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-218e4d55-8d29-4110-819d-fdd711ece3e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:50:23.565775 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1183/2346] 10.222.0.42 () {58 vars in 1018 bytes} [Thu Jun 16 17:50:23 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:50:23.577878 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383fcfe0-9d02-44c4-bb52-7b497d922f93 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:23.579530 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-383fcfe0-9d02-44c4-bb52-7b497d922f93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:50:23.655040 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383fcfe0-9d02-44c4-bb52-7b497d922f93 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:23.658662 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fec2d16-a490-4864-8464-4b75f5e880f6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd89029c1b3134df1962c041988bead74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:23.659302 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fec2d16-a490-4864-8464-4b75f5e880f6 service glance] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:23.659649 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fec2d16-a490-4864-8464-4b75f5e880f6 service glance] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:23.661093 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fec2d16-a490-4864-8464-4b75f5e880f6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:23.671925 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fec2d16-a490-4864-8464-4b75f5e880f6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:23.673375 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fec2d16-a490-4864-8464-4b75f5e880f6 service glance] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:23.685127 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1164/2347] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:50:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:50:23.757740 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421d76a5-d88c-4032-8914-9d94e6ad6a96 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:23.758143 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421d76a5-d88c-4032-8914-9d94e6ad6a96 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:23.758492 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421d76a5-d88c-4032-8914-9d94e6ad6a96 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:23.761387 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1184/2348] 10.222.0.42 () {54 vars in 921 bytes} [Thu Jun 16 17:50:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:50:23.766363 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca6739d-7f6f-40ea-b823-e8c51db1337f None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:23.766735 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca6739d-7f6f-40ea-b823-e8c51db1337f None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:23.767099 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca6739d-7f6f-40ea-b823-e8c51db1337f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:23.787825 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-aca6739d-7f6f-40ea-b823-e8c51db1337f None None] MFA Rules not processed for user `f94416103f6245888214a997b014f47a`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:50:23.789072 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-aca6739d-7f6f-40ea-b823-e8c51db1337f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:50:23.853843 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1165/2349] 10.222.0.42 () {58 vars in 1018 bytes} [Thu Jun 16 17:50:23 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:50:23.869319 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421d76a5-d88c-4032-8914-9d94e6ad6a96 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:23.871292 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-421d76a5-d88c-4032-8914-9d94e6ad6a96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:50:23.895779 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421d76a5-d88c-4032-8914-9d94e6ad6a96 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:23.900437 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184c4a25-b284-4b34-a75c-ab4a5dfd9c0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:23.901194 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184c4a25-b284-4b34-a75c-ab4a5dfd9c0c service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:23.901547 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184c4a25-b284-4b34-a75c-ab4a5dfd9c0c service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:23.901900 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184c4a25-b284-4b34-a75c-ab4a5dfd9c0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:23.911506 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184c4a25-b284-4b34-a75c-ab4a5dfd9c0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:23.913240 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184c4a25-b284-4b34-a75c-ab4a5dfd9c0c service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:23.922687 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1185/2350] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:50:23 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:50:24.030920 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16fd172-fa5c-4df0-91f6-469f53553db2 None None] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:24.031227 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16fd172-fa5c-4df0-91f6-469f53553db2 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:24.031572 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16fd172-fa5c-4df0-91f6-469f53553db2 None None] PATH_INFO: `/` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:24.033090 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1166/2351] 10.222.0.42 () {54 vars in 921 bytes} [Thu Jun 16 17:50:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:50:24.039496 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f2ed84-c1f4-4422-9f73-22a4ef9b691b None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:24.040011 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f2ed84-c1f4-4422-9f73-22a4ef9b691b None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:24.040306 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f2ed84-c1f4-4422-9f73-22a4ef9b691b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:24.059697 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-03f2ed84-c1f4-4422-9f73-22a4ef9b691b None None] MFA Rules not processed for user `f86c16179a70474a817be9fda33ece9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:50:24.060946 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-03f2ed84-c1f4-4422-9f73-22a4ef9b691b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:50:24.071486 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1186/2352] 10.222.0.42 () {58 vars in 1018 bytes} [Thu Jun 16 17:50:24 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:50:24.082659 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16fd172-fa5c-4df0-91f6-469f53553db2 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:24.084259 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-e16fd172-fa5c-4df0-91f6-469f53553db2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:50:24.102029 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16fd172-fa5c-4df0-91f6-469f53553db2 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:24.105580 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785caec8-fa63-4d07-af72-7f9035280a3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:24.106214 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785caec8-fa63-4d07-af72-7f9035280a3b service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:24.106501 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785caec8-fa63-4d07-af72-7f9035280a3b service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:24.106779 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785caec8-fa63-4d07-af72-7f9035280a3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:24.117106 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785caec8-fa63-4d07-af72-7f9035280a3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:24.118560 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785caec8-fa63-4d07-af72-7f9035280a3b service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:24.129382 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1167/2353] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:50:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:50:27.229405 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a2b9c4-7177-46ca-aab7-e75f0df95923 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:27.230443 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a2b9c4-7177-46ca-aab7-e75f0df95923 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:27.230890 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a2b9c4-7177-46ca-aab7-e75f0df95923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:27.267935 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-62a2b9c4-7177-46ca-aab7-e75f0df95923 None None] MFA Rules not processed for user `71443c30b4554719b5549493bf3cc00c`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:50:27.270876 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-62a2b9c4-7177-46ca-aab7-e75f0df95923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:50:27.300569 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1187/2354] 10.222.0.42 () {58 vars in 1009 bytes} [Thu Jun 16 17:50:27 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:50:27.307201 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785caec8-fa63-4d07-af72-7f9035280a3b service neutron] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:27.327651 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785caec8-fa63-4d07-af72-7f9035280a3b service neutron] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:27.334785 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3626bfca-55c2-481e-bf3d-e5b48e7b1891 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47b7a9cd615c41d6983a0007b475038c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:27.335483 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3626bfca-55c2-481e-bf3d-e5b48e7b1891 service swift] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:27.335920 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3626bfca-55c2-481e-bf3d-e5b48e7b1891 service swift] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:27.336257 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3626bfca-55c2-481e-bf3d-e5b48e7b1891 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:27.349129 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3626bfca-55c2-481e-bf3d-e5b48e7b1891 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:27.350667 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3626bfca-55c2-481e-bf3d-e5b48e7b1891 service swift] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:27.360208 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1168/2355] 10.222.0.42 () {60 vars in 1384 bytes} [Thu Jun 16 17:50:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 17:50:34.402288 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a2b9c4-7177-46ca-aab7-e75f0df95923 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:34.412056 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a2b9c4-7177-46ca-aab7-e75f0df95923 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:34.415612 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2565d9b7-822a-46ae-aac9-2eb7e322466c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:34.416247 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2565d9b7-822a-46ae-aac9-2eb7e322466c service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:34.416557 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2565d9b7-822a-46ae-aac9-2eb7e322466c service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:34.416831 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2565d9b7-822a-46ae-aac9-2eb7e322466c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:34.418659 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-2565d9b7-822a-46ae-aac9-2eb7e322466c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:50:34.497418 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2565d9b7-822a-46ae-aac9-2eb7e322466c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:34.499168 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2565d9b7-822a-46ae-aac9-2eb7e322466c service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:34.510378 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1188/2356] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:50:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:50:37.356401 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=61 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:50:37.961130 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3626bfca-55c2-481e-bf3d-e5b48e7b1891 service swift] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:37.962747 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-3626bfca-55c2-481e-bf3d-e5b48e7b1891 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:50:38.026454 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2565d9b7-822a-46ae-aac9-2eb7e322466c service neutron] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:38.028159 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-2565d9b7-822a-46ae-aac9-2eb7e322466c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:50:38.042983 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2565d9b7-822a-46ae-aac9-2eb7e322466c service neutron] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:38.047012 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3626bfca-55c2-481e-bf3d-e5b48e7b1891 service swift] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:38.047884 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725ed383-3944-4479-9725-e5f117cbd820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:38.048807 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725ed383-3944-4479-9725-e5f117cbd820 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:38.049157 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725ed383-3944-4479-9725-e5f117cbd820 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:38.049356 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725ed383-3944-4479-9725-e5f117cbd820 admin admin] PATH_INFO: `/v3/users/813dd0cf7e1d4724b903b358aaac1ea7` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:38.052256 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725ed383-3944-4479-9725-e5f117cbd820 admin admin] RBAC: Authorizing `identity:delete_user(user_id=813dd0cf7e1d4724b903b358aaac1ea7)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:38.053616 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2415fb6c-09b6-4d9d-896f-4728b30dffc8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0570d8d9508d4546ae897276f7d181d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:38.054270 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2415fb6c-09b6-4d9d-896f-4728b30dffc8 service placement] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:38.054452 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725ed383-3944-4479-9725-e5f117cbd820 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:38.054615 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2415fb6c-09b6-4d9d-896f-4728b30dffc8 service placement] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:38.055149 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2415fb6c-09b6-4d9d-896f-4728b30dffc8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:38.056970 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-2415fb6c-09b6-4d9d-896f-4728b30dffc8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:50:38.076855 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2415fb6c-09b6-4d9d-896f-4728b30dffc8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:38.078469 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2415fb6c-09b6-4d9d-896f-4728b30dffc8 service placement] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:38.090497 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1169/2357] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:50:37 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:50:38.113426 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-725ed383-3944-4479-9725-e5f117cbd820 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '813dd0cf7e1d4724b903b358aaac1ea7'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:38.119272 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:50:38.128778 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-725ed383-3944-4479-9725-e5f117cbd820 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '813dd0cf7e1d4724b903b358aaac1ea7'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:38.129528 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-725ed383-3944-4479-9725-e5f117cbd820 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '813dd0cf7e1d4724b903b358aaac1ea7'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:38.137815 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-725ed383-3944-4479-9725-e5f117cbd820 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '813dd0cf7e1d4724b903b358aaac1ea7'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:38.144799 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-725ed383-3944-4479-9725-e5f117cbd820 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '813dd0cf7e1d4724b903b358aaac1ea7'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:38.150684 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1189/2358] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jun 16 17:50:38 2022] DELETE /identity/v3/users/813dd0cf7e1d4724b903b358aaac1ea7 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:50:38.332079 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2415fb6c-09b6-4d9d-896f-4728b30dffc8 service placement] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:38.343494 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2415fb6c-09b6-4d9d-896f-4728b30dffc8 service placement] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:38.403057 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7842b26b-1e57-4559-b04d-4ca80585f391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:38.403920 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7842b26b-1e57-4559-b04d-4ca80585f391 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:38.404206 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7842b26b-1e57-4559-b04d-4ca80585f391 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:38.404526 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7842b26b-1e57-4559-b04d-4ca80585f391 admin admin] PATH_INFO: `/v3/projects/dd279f49cbd84e9f926aa6372a4728d4` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:38.405706 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7842b26b-1e57-4559-b04d-4ca80585f391 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd279f49cbd84e9f926aa6372a4728d4)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:38.407613 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7842b26b-1e57-4559-b04d-4ca80585f391 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:38.454322 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-7842b26b-1e57-4559-b04d-4ca80585f391 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd279f49cbd84e9f926aa6372a4728d4', 'request_id': 'req-7842b26b-1e57-4559-b04d-4ca80585f391'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:38.462494 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-7842b26b-1e57-4559-b04d-4ca80585f391 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd279f49cbd84e9f926aa6372a4728d4', 'request_id': 'req-7842b26b-1e57-4559-b04d-4ca80585f391'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:38.468432 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-7842b26b-1e57-4559-b04d-4ca80585f391 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd279f49cbd84e9f926aa6372a4728d4', 'request_id': 'req-7842b26b-1e57-4559-b04d-4ca80585f391'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:38.479932 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-7842b26b-1e57-4559-b04d-4ca80585f391 admin admin] The token cache is being invalidate because project dd279f49cbd84e9f926aa6372a4728d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80357) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:50:38.480215 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-7842b26b-1e57-4559-b04d-4ca80585f391 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:38.480901 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-7842b26b-1e57-4559-b04d-4ca80585f391 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:38.482452 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1170/2359] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jun 16 17:50:38 2022] DELETE /identity/v3/projects/dd279f49cbd84e9f926aa6372a4728d4 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:50:38.487819 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae139410-2329-4214-8033-f1e1c8451c27 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:38.488120 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae139410-2329-4214-8033-f1e1c8451c27 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:38.488418 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae139410-2329-4214-8033-f1e1c8451c27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:38.507555 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-ae139410-2329-4214-8033-f1e1c8451c27 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:50:38.508782 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-ae139410-2329-4214-8033-f1e1c8451c27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:50:38.570269 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1190/2360] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:50:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:50:38.578337 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef52c45c-3da8-428c-9760-11c2c2eecf32 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:38.578633 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef52c45c-3da8-428c-9760-11c2c2eecf32 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:38.578917 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef52c45c-3da8-428c-9760-11c2c2eecf32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:38.597966 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-ef52c45c-3da8-428c-9760-11c2c2eecf32 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:50:38.599178 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-ef52c45c-3da8-428c-9760-11c2c2eecf32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:50:38.607626 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1171/2361] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:50:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:50:38.611855 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae139410-2329-4214-8033-f1e1c8451c27 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:38.622305 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae139410-2329-4214-8033-f1e1c8451c27 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:38.627031 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d096434-5e42-4d38-aef1-298dc25f1859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:38.627697 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d096434-5e42-4d38-aef1-298dc25f1859 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:38.628754 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d096434-5e42-4d38-aef1-298dc25f1859 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:38.629229 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d096434-5e42-4d38-aef1-298dc25f1859 admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:38.630137 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d096434-5e42-4d38-aef1-298dc25f1859 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:50:38.630453 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d096434-5e42-4d38-aef1-298dc25f1859 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:38.632059 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d096434-5e42-4d38-aef1-298dc25f1859 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:38.644829 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1191/2362] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jun 16 17:50:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:50:38.649101 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef52c45c-3da8-428c-9760-11c2c2eecf32 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:38.659975 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef52c45c-3da8-428c-9760-11c2c2eecf32 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:38.664785 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9d91f6-db07-475f-b646-24a368c3fdd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:38.665444 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9d91f6-db07-475f-b646-24a368c3fdd3 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:38.665730 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9d91f6-db07-475f-b646-24a368c3fdd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:38.666007 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9d91f6-db07-475f-b646-24a368c3fdd3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:38.666692 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9d91f6-db07-475f-b646-24a368c3fdd3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:38.668657 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9d91f6-db07-475f-b646-24a368c3fdd3 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:38.679385 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:50:38.697177 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1172/2363] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jun 16 17:50:38 2022] POST /identity/v3/projects => generated 371 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:50:38.701163 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d096434-5e42-4d38-aef1-298dc25f1859 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:38.711330 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d096434-5e42-4d38-aef1-298dc25f1859 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:38.769044 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dcc756-c2e1-43c6-ab2b-55fca925991c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:38.769700 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dcc756-c2e1-43c6-ab2b-55fca925991c admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:38.769986 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dcc756-c2e1-43c6-ab2b-55fca925991c admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:38.770266 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dcc756-c2e1-43c6-ab2b-55fca925991c admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:38.770963 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dcc756-c2e1-43c6-ab2b-55fca925991c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:38.773028 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dcc756-c2e1-43c6-ab2b-55fca925991c admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:38.792999 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1192/2364] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jun 16 17:50:38 2022] POST /identity/v3/users => generated 286 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:50:38.797017 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9d91f6-db07-475f-b646-24a368c3fdd3 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:38.807961 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9d91f6-db07-475f-b646-24a368c3fdd3 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:38.812580 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8a36f9-a962-4314-a57a-43c5c6bebce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:38.813282 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8a36f9-a962-4314-a57a-43c5c6bebce9 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:38.813566 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8a36f9-a962-4314-a57a-43c5c6bebce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:38.813843 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8a36f9-a962-4314-a57a-43c5c6bebce9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:38.814559 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8a36f9-a962-4314-a57a-43c5c6bebce9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:38.816178 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8a36f9-a962-4314-a57a-43c5c6bebce9 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:38.826571 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1173/2365] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:50:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:50:38.830425 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dcc756-c2e1-43c6-ab2b-55fca925991c admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:38.840882 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dcc756-c2e1-43c6-ab2b-55fca925991c admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:38.845464 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054e93ca-4f63-453f-b2e8-33f46b444361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:38.846118 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054e93ca-4f63-453f-b2e8-33f46b444361 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:38.846402 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054e93ca-4f63-453f-b2e8-33f46b444361 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:38.847271 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054e93ca-4f63-453f-b2e8-33f46b444361 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:38.848219 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054e93ca-4f63-453f-b2e8-33f46b444361 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:38.849974 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054e93ca-4f63-453f-b2e8-33f46b444361 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:38.863860 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1193/2366] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:50:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:50:38.865364 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8a36f9-a962-4314-a57a-43c5c6bebce9 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:38.875962 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8a36f9-a962-4314-a57a-43c5c6bebce9 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:38.881479 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87c4a5e-bc5c-46a2-9763-db45424d90a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:38.882385 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87c4a5e-bc5c-46a2-9763-db45424d90a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:38.882665 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87c4a5e-bc5c-46a2-9763-db45424d90a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:38.882938 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87c4a5e-bc5c-46a2-9763-db45424d90a2 admin admin] PATH_INFO: `/v3/projects/640525dc121f4bea99dd38ead3f0c00f/users/63ea7aeb08cd4e5bab0a8c7566e8c258/roles/0be2075946f54c8485f1c275f746478a` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:38.900856 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87c4a5e-bc5c-46a2-9763-db45424d90a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=640525dc121f4bea99dd38ead3f0c00f, user_id=63ea7aeb08cd4e5bab0a8c7566e8c258, role_id=0be2075946f54c8485f1c275f746478a)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:38.903499 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87c4a5e-bc5c-46a2-9763-db45424d90a2 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:38.918613 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1174/2367] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:50:38 2022] PUT /identity/v3/projects/640525dc121f4bea99dd38ead3f0c00f/users/63ea7aeb08cd4e5bab0a8c7566e8c258/roles/0be2075946f54c8485f1c275f746478a => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:50:38.930205 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793743bc-57da-4819-b9a5-d538ac46dc9a None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:38.930506 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793743bc-57da-4819-b9a5-d538ac46dc9a None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:38.930790 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793743bc-57da-4819-b9a5-d538ac46dc9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:38.962160 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-793743bc-57da-4819-b9a5-d538ac46dc9a None None] MFA Rules not processed for user `63ea7aeb08cd4e5bab0a8c7566e8c258`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:50:38.963465 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-793743bc-57da-4819-b9a5-d538ac46dc9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:50:38.995901 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:50:39.044493 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1194/2368] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:50:38 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:50:39.048917 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87c4a5e-bc5c-46a2-9763-db45424d90a2 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:39.134392 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87c4a5e-bc5c-46a2-9763-db45424d90a2 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:39.144026 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fff7e9-9dd8-4a25-ad98-0df83883a4ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:39.144787 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fff7e9-9dd8-4a25-ad98-0df83883a4ae admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:39.144983 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fff7e9-9dd8-4a25-ad98-0df83883a4ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:39.145223 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fff7e9-9dd8-4a25-ad98-0df83883a4ae admin admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:39.145920 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fff7e9-9dd8-4a25-ad98-0df83883a4ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:39.147960 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fff7e9-9dd8-4a25-ad98-0df83883a4ae admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:39.178620 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1175/2369] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jun 16 17:50:39 2022] POST /identity/v3/projects => generated 371 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:50:39.182969 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793743bc-57da-4819-b9a5-d538ac46dc9a None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:39.198981 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793743bc-57da-4819-b9a5-d538ac46dc9a None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:39.261893 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89b6efb-0441-41f3-bd83-269adee8cdd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:39.262565 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89b6efb-0441-41f3-bd83-269adee8cdd6 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:39.263121 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89b6efb-0441-41f3-bd83-269adee8cdd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:39.263206 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89b6efb-0441-41f3-bd83-269adee8cdd6 admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:39.263820 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89b6efb-0441-41f3-bd83-269adee8cdd6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:39.265756 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89b6efb-0441-41f3-bd83-269adee8cdd6 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:39.287858 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1195/2370] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jun 16 17:50:39 2022] POST /identity/v3/users => generated 292 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:50:39.290216 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fff7e9-9dd8-4a25-ad98-0df83883a4ae admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:39.301198 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fff7e9-9dd8-4a25-ad98-0df83883a4ae admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:39.305781 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21506100-c080-43c1-9270-02484d31246c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:39.306430 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21506100-c080-43c1-9270-02484d31246c admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:39.306714 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21506100-c080-43c1-9270-02484d31246c admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:39.306988 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21506100-c080-43c1-9270-02484d31246c admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:39.307703 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21506100-c080-43c1-9270-02484d31246c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:39.309323 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21506100-c080-43c1-9270-02484d31246c admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:39.320492 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1176/2371] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:50:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:50:39.324593 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89b6efb-0441-41f3-bd83-269adee8cdd6 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:39.335048 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89b6efb-0441-41f3-bd83-269adee8cdd6 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:39.339608 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d41994-b7fa-4347-a247-6baca698b5d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:39.340608 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d41994-b7fa-4347-a247-6baca698b5d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:39.341128 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d41994-b7fa-4347-a247-6baca698b5d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:39.341413 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d41994-b7fa-4347-a247-6baca698b5d5 admin admin] PATH_INFO: `/v3/projects/60556507ddc841efa7d9b632a0109ad1/users/d15d1d1094d44f70beaf20661c989b26/roles/3ca2dcbbd29d4a31a3ab78f2299bc239` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:39.357277 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d41994-b7fa-4347-a247-6baca698b5d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60556507ddc841efa7d9b632a0109ad1, user_id=d15d1d1094d44f70beaf20661c989b26, role_id=3ca2dcbbd29d4a31a3ab78f2299bc239)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:39.359919 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d41994-b7fa-4347-a247-6baca698b5d5 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:39.375362 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1196/2372] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:50:39 2022] PUT /identity/v3/projects/60556507ddc841efa7d9b632a0109ad1/users/d15d1d1094d44f70beaf20661c989b26/roles/3ca2dcbbd29d4a31a3ab78f2299bc239 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:50:39.380029 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21506100-c080-43c1-9270-02484d31246c admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:39.390493 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21506100-c080-43c1-9270-02484d31246c admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:39.448756 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfdab12-76b2-4df3-ae5e-5a0f7808a03c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:39.449675 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfdab12-76b2-4df3-ae5e-5a0f7808a03c admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:39.450121 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfdab12-76b2-4df3-ae5e-5a0f7808a03c admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:39.450651 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfdab12-76b2-4df3-ae5e-5a0f7808a03c admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:39.451750 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfdab12-76b2-4df3-ae5e-5a0f7808a03c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:39.454045 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfdab12-76b2-4df3-ae5e-5a0f7808a03c admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:39.465515 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1177/2373] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:50:39 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:50:39.469940 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d41994-b7fa-4347-a247-6baca698b5d5 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:39.481320 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d41994-b7fa-4347-a247-6baca698b5d5 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:39.486203 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1186f50b-d4ad-4099-9bc8-1b338e241309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:39.487086 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1186f50b-d4ad-4099-9bc8-1b338e241309 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:39.488078 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1186f50b-d4ad-4099-9bc8-1b338e241309 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:39.489129 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1186f50b-d4ad-4099-9bc8-1b338e241309 admin admin] PATH_INFO: `/v3/projects/60556507ddc841efa7d9b632a0109ad1/users/d15d1d1094d44f70beaf20661c989b26/roles/3ca2dcbbd29d4a31a3ab78f2299bc239` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:39.491557 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1186f50b-d4ad-4099-9bc8-1b338e241309 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60556507ddc841efa7d9b632a0109ad1, user_id=d15d1d1094d44f70beaf20661c989b26, role_id=3ca2dcbbd29d4a31a3ab78f2299bc239)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:39.494611 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1186f50b-d4ad-4099-9bc8-1b338e241309 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:39.505753 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1197/2374] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:50:39 2022] PUT /identity/v3/projects/60556507ddc841efa7d9b632a0109ad1/users/d15d1d1094d44f70beaf20661c989b26/roles/3ca2dcbbd29d4a31a3ab78f2299bc239 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:50:39.516801 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff5b38e-2528-46ef-b86d-96cdbb7b77c9 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:39.517274 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff5b38e-2528-46ef-b86d-96cdbb7b77c9 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:39.517445 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff5b38e-2528-46ef-b86d-96cdbb7b77c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:39.546294 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-6ff5b38e-2528-46ef-b86d-96cdbb7b77c9 None None] MFA Rules not processed for user `d15d1d1094d44f70beaf20661c989b26`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:50:39.547528 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-6ff5b38e-2528-46ef-b86d-96cdbb7b77c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:50:39.561291 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:50:39.620328 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1178/2375] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:50:39 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:50:39.634296 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1186f50b-d4ad-4099-9bc8-1b338e241309 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:39.636610 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-1186f50b-d4ad-4099-9bc8-1b338e241309 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:50:39.699503 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1186f50b-d4ad-4099-9bc8-1b338e241309 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:39.703785 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f5f416-c152-47ed-8b4d-677365173915 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24d5e2637b7043e08313909d5b06e329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:39.704459 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f5f416-c152-47ed-8b4d-677365173915 service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:39.704742 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f5f416-c152-47ed-8b4d-677365173915 service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:39.705029 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f5f416-c152-47ed-8b4d-677365173915 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:39.714956 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f5f416-c152-47ed-8b4d-677365173915 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:39.716393 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f5f416-c152-47ed-8b4d-677365173915 service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:39.726901 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1198/2376] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:50:39 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:50:40.130242 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff5b38e-2528-46ef-b86d-96cdbb7b77c9 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:40.193983 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff5b38e-2528-46ef-b86d-96cdbb7b77c9 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:40.198488 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abda58f4-477f-4f2f-863d-44a74aeb2b01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:40.199347 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abda58f4-477f-4f2f-863d-44a74aeb2b01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:40.199633 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abda58f4-477f-4f2f-863d-44a74aeb2b01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:40.199901 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abda58f4-477f-4f2f-863d-44a74aeb2b01 admin admin] PATH_INFO: `/v3/users/63ea7aeb08cd4e5bab0a8c7566e8c258` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:40.201128 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abda58f4-477f-4f2f-863d-44a74aeb2b01 admin admin] RBAC: Authorizing `identity:delete_user(user_id=63ea7aeb08cd4e5bab0a8c7566e8c258)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:40.203029 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abda58f4-477f-4f2f-863d-44a74aeb2b01 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:40.263122 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-abda58f4-477f-4f2f-863d-44a74aeb2b01 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63ea7aeb08cd4e5bab0a8c7566e8c258'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:40.263846 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-abda58f4-477f-4f2f-863d-44a74aeb2b01 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63ea7aeb08cd4e5bab0a8c7566e8c258'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:40.273425 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-abda58f4-477f-4f2f-863d-44a74aeb2b01 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63ea7aeb08cd4e5bab0a8c7566e8c258'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:40.274359 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-abda58f4-477f-4f2f-863d-44a74aeb2b01 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63ea7aeb08cd4e5bab0a8c7566e8c258'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:40.288664 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-abda58f4-477f-4f2f-863d-44a74aeb2b01 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63ea7aeb08cd4e5bab0a8c7566e8c258'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:40.305698 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1179/2377] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jun 16 17:50:40 2022] DELETE /identity/v3/users/63ea7aeb08cd4e5bab0a8c7566e8c258 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:50:40.310587 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41d729d-f3ba-478e-a052-0c9557789908 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:40.311050 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41d729d-f3ba-478e-a052-0c9557789908 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:40.311385 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41d729d-f3ba-478e-a052-0c9557789908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:40.332566 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-a41d729d-f3ba-478e-a052-0c9557789908 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:50:40.333784 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-a41d729d-f3ba-478e-a052-0c9557789908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:50:40.351438 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:50:40.402607 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1199/2378] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:50:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:50:40.413859 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abda58f4-477f-4f2f-863d-44a74aeb2b01 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:40.415275 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-abda58f4-477f-4f2f-863d-44a74aeb2b01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:50:40.457748 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:50:40.484677 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abda58f4-477f-4f2f-863d-44a74aeb2b01 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:40.489417 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199b6e4f-e35d-4a25-ba1e-1f4f31e68140 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:40.490366 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199b6e4f-e35d-4a25-ba1e-1f4f31e68140 service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:40.490694 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199b6e4f-e35d-4a25-ba1e-1f4f31e68140 service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:40.491106 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199b6e4f-e35d-4a25-ba1e-1f4f31e68140 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:40.503582 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199b6e4f-e35d-4a25-ba1e-1f4f31e68140 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:40.505515 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199b6e4f-e35d-4a25-ba1e-1f4f31e68140 service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:40.516212 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1180/2379] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:50:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:50:40.804392 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41d729d-f3ba-478e-a052-0c9557789908 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:40.805915 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-a41d729d-f3ba-478e-a052-0c9557789908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:50:40.819324 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41d729d-f3ba-478e-a052-0c9557789908 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:40.824278 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd77607-7070-4a31-883f-94f0de87c813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:40.825438 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd77607-7070-4a31-883f-94f0de87c813 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:40.825719 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd77607-7070-4a31-883f-94f0de87c813 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:40.825993 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd77607-7070-4a31-883f-94f0de87c813 admin admin] PATH_INFO: `/v3/projects/640525dc121f4bea99dd38ead3f0c00f` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:40.827248 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd77607-7070-4a31-883f-94f0de87c813 admin admin] RBAC: Authorizing `identity:delete_project(project_id=640525dc121f4bea99dd38ead3f0c00f)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:40.829210 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd77607-7070-4a31-883f-94f0de87c813 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:40.879247 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-6bd77607-7070-4a31-883f-94f0de87c813 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '640525dc121f4bea99dd38ead3f0c00f', 'request_id': 'req-6bd77607-7070-4a31-883f-94f0de87c813'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:40.888615 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-6bd77607-7070-4a31-883f-94f0de87c813 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '640525dc121f4bea99dd38ead3f0c00f', 'request_id': 'req-6bd77607-7070-4a31-883f-94f0de87c813'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:40.895446 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-6bd77607-7070-4a31-883f-94f0de87c813 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '640525dc121f4bea99dd38ead3f0c00f', 'request_id': 'req-6bd77607-7070-4a31-883f-94f0de87c813'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:40.905874 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-6bd77607-7070-4a31-883f-94f0de87c813 admin admin] The token cache is being invalidate because project 640525dc121f4bea99dd38ead3f0c00f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:50:40.906164 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-6bd77607-7070-4a31-883f-94f0de87c813 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:40.906819 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-6bd77607-7070-4a31-883f-94f0de87c813 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:40.908383 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1200/2380] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jun 16 17:50:40 2022] DELETE /identity/v3/projects/640525dc121f4bea99dd38ead3f0c00f => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:50:40.912467 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199b6e4f-e35d-4a25-ba1e-1f4f31e68140 service neutron] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:40.922164 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199b6e4f-e35d-4a25-ba1e-1f4f31e68140 service neutron] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:40.984645 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f42460-1ef8-4467-a4a2-97c4c554c854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:40.985303 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f42460-1ef8-4467-a4a2-97c4c554c854 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:40.986171 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f42460-1ef8-4467-a4a2-97c4c554c854 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:40.986463 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f42460-1ef8-4467-a4a2-97c4c554c854 admin admin] PATH_INFO: `/v3/users/d15d1d1094d44f70beaf20661c989b26` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:40.988325 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f42460-1ef8-4467-a4a2-97c4c554c854 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d15d1d1094d44f70beaf20661c989b26)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:40.991098 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f42460-1ef8-4467-a4a2-97c4c554c854 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:41.056710 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-68f42460-1ef8-4467-a4a2-97c4c554c854 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd15d1d1094d44f70beaf20661c989b26'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:41.057405 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-68f42460-1ef8-4467-a4a2-97c4c554c854 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd15d1d1094d44f70beaf20661c989b26'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:41.064946 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-68f42460-1ef8-4467-a4a2-97c4c554c854 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd15d1d1094d44f70beaf20661c989b26'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:41.065695 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-68f42460-1ef8-4467-a4a2-97c4c554c854 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd15d1d1094d44f70beaf20661c989b26'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:41.078189 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-68f42460-1ef8-4467-a4a2-97c4c554c854 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd15d1d1094d44f70beaf20661c989b26'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:41.091875 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1181/2381] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jun 16 17:50:40 2022] DELETE /identity/v3/users/d15d1d1094d44f70beaf20661c989b26 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:50:41.385918 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd77607-7070-4a31-883f-94f0de87c813 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:41.387576 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-6bd77607-7070-4a31-883f-94f0de87c813 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:50:41.401336 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd77607-7070-4a31-883f-94f0de87c813 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:41.463762 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3017cbc-06eb-485e-8955-e59d8d9347e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:41.464582 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3017cbc-06eb-485e-8955-e59d8d9347e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:41.464869 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3017cbc-06eb-485e-8955-e59d8d9347e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:41.465149 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3017cbc-06eb-485e-8955-e59d8d9347e6 admin admin] PATH_INFO: `/v3/projects/60556507ddc841efa7d9b632a0109ad1` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:41.466348 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3017cbc-06eb-485e-8955-e59d8d9347e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=60556507ddc841efa7d9b632a0109ad1)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:41.468246 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3017cbc-06eb-485e-8955-e59d8d9347e6 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:41.516306 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:50:41.517712 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-d3017cbc-06eb-485e-8955-e59d8d9347e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60556507ddc841efa7d9b632a0109ad1', 'request_id': 'req-d3017cbc-06eb-485e-8955-e59d8d9347e6'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:41.526376 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-d3017cbc-06eb-485e-8955-e59d8d9347e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60556507ddc841efa7d9b632a0109ad1', 'request_id': 'req-d3017cbc-06eb-485e-8955-e59d8d9347e6'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:41.533623 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-d3017cbc-06eb-485e-8955-e59d8d9347e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60556507ddc841efa7d9b632a0109ad1', 'request_id': 'req-d3017cbc-06eb-485e-8955-e59d8d9347e6'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:41.544998 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-d3017cbc-06eb-485e-8955-e59d8d9347e6 admin admin] The token cache is being invalidate because project 60556507ddc841efa7d9b632a0109ad1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:50:41.545292 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-d3017cbc-06eb-485e-8955-e59d8d9347e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:41.545925 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-d3017cbc-06eb-485e-8955-e59d8d9347e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:41.547458 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1201/2382] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jun 16 17:50:41 2022] DELETE /identity/v3/projects/60556507ddc841efa7d9b632a0109ad1 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:50:41.553531 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c058b31-9413-444c-94a9-f438a8079a3d None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:41.553996 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c058b31-9413-444c-94a9-f438a8079a3d None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:41.554256 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c058b31-9413-444c-94a9-f438a8079a3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:41.571640 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-9c058b31-9413-444c-94a9-f438a8079a3d None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:50:41.572876 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-9c058b31-9413-444c-94a9-f438a8079a3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:50:41.624499 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:50:41.641813 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1182/2383] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:50:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:50:41.650351 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4769d7f9-b0cb-4429-9112-f2ef2467aa54 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:41.650679 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4769d7f9-b0cb-4429-9112-f2ef2467aa54 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:41.650961 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4769d7f9-b0cb-4429-9112-f2ef2467aa54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:41.670779 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-4769d7f9-b0cb-4429-9112-f2ef2467aa54 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:50:41.671988 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-4769d7f9-b0cb-4429-9112-f2ef2467aa54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:50:41.680782 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1202/2384] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:50:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:50:41.685677 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c058b31-9413-444c-94a9-f438a8079a3d None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:41.696103 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c058b31-9413-444c-94a9-f438a8079a3d None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:41.700819 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebc7d64-aa56-4f88-8781-78e8c2bd1039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:41.701478 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebc7d64-aa56-4f88-8781-78e8c2bd1039 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:41.701755 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebc7d64-aa56-4f88-8781-78e8c2bd1039 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:41.702028 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebc7d64-aa56-4f88-8781-78e8c2bd1039 admin admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:41.702751 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebc7d64-aa56-4f88-8781-78e8c2bd1039 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:50:41.703055 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebc7d64-aa56-4f88-8781-78e8c2bd1039 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:41.704670 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebc7d64-aa56-4f88-8781-78e8c2bd1039 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:41.717624 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1183/2385] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jun 16 17:50:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:50:41.722421 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4769d7f9-b0cb-4429-9112-f2ef2467aa54 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:41.732785 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4769d7f9-b0cb-4429-9112-f2ef2467aa54 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:41.738793 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ae183c-b8e4-431d-a406-79d1dfea3d75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:41.739449 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ae183c-b8e4-431d-a406-79d1dfea3d75 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:41.739730 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ae183c-b8e4-431d-a406-79d1dfea3d75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:41.740116 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ae183c-b8e4-431d-a406-79d1dfea3d75 admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:41.740934 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ae183c-b8e4-431d-a406-79d1dfea3d75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:41.743903 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ae183c-b8e4-431d-a406-79d1dfea3d75 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:41.776236 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1203/2386] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jun 16 17:50:41 2022] POST /identity/v3/projects => generated 387 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:50:41.780448 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebc7d64-aa56-4f88-8781-78e8c2bd1039 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:41.791540 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebc7d64-aa56-4f88-8781-78e8c2bd1039 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:41.854190 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4562d60-df63-4f5b-87de-6736f9867f4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:41.854947 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4562d60-df63-4f5b-87de-6736f9867f4d admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:41.855276 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4562d60-df63-4f5b-87de-6736f9867f4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:41.855630 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4562d60-df63-4f5b-87de-6736f9867f4d admin admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:41.856459 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4562d60-df63-4f5b-87de-6736f9867f4d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:41.858682 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4562d60-df63-4f5b-87de-6736f9867f4d admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:41.896994 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1184/2387] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jun 16 17:50:41 2022] POST /identity/v3/users => generated 294 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:50:41.904557 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ae183c-b8e4-431d-a406-79d1dfea3d75 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:41.921443 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ae183c-b8e4-431d-a406-79d1dfea3d75 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:41.926449 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb523e5-5f50-498d-be57-fe95b7f61291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:41.927211 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb523e5-5f50-498d-be57-fe95b7f61291 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:41.927497 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb523e5-5f50-498d-be57-fe95b7f61291 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:41.927771 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb523e5-5f50-498d-be57-fe95b7f61291 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:41.928629 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb523e5-5f50-498d-be57-fe95b7f61291 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:41.930235 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb523e5-5f50-498d-be57-fe95b7f61291 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:41.943163 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1204/2388] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:50:41 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:50:41.947183 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4562d60-df63-4f5b-87de-6736f9867f4d admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:41.959215 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4562d60-df63-4f5b-87de-6736f9867f4d admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:41.964467 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee338ee0-c953-482a-bb2c-cac930376469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:41.965148 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee338ee0-c953-482a-bb2c-cac930376469 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:41.965508 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee338ee0-c953-482a-bb2c-cac930376469 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:41.965861 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee338ee0-c953-482a-bb2c-cac930376469 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:41.966663 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee338ee0-c953-482a-bb2c-cac930376469 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:41.968492 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee338ee0-c953-482a-bb2c-cac930376469 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:41.983045 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1185/2389] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:50:41 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 17:50:41.985951 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb523e5-5f50-498d-be57-fe95b7f61291 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:41.996789 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb523e5-5f50-498d-be57-fe95b7f61291 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:42.001381 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ad2684-fb5e-4adf-bd8a-cca3c7c172f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:42.002244 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ad2684-fb5e-4adf-bd8a-cca3c7c172f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:42.002523 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ad2684-fb5e-4adf-bd8a-cca3c7c172f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:42.002838 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ad2684-fb5e-4adf-bd8a-cca3c7c172f5 admin admin] PATH_INFO: `/v3/projects/c7460e48168d4a75a18b667ac5baf0a0/users/6e117d9bed4b41aebf89018e48e12dd7/roles/0be2075946f54c8485f1c275f746478a` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:42.017619 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ad2684-fb5e-4adf-bd8a-cca3c7c172f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7460e48168d4a75a18b667ac5baf0a0, user_id=6e117d9bed4b41aebf89018e48e12dd7, role_id=0be2075946f54c8485f1c275f746478a)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:42.020489 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ad2684-fb5e-4adf-bd8a-cca3c7c172f5 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:42.033634 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1205/2390] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:50:41 2022] PUT /identity/v3/projects/c7460e48168d4a75a18b667ac5baf0a0/users/6e117d9bed4b41aebf89018e48e12dd7/roles/0be2075946f54c8485f1c275f746478a => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:50:42.045961 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816f7185-3940-4be7-abaa-2666e303091b None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:42.046255 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816f7185-3940-4be7-abaa-2666e303091b None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:42.046534 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816f7185-3940-4be7-abaa-2666e303091b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:42.073850 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-816f7185-3940-4be7-abaa-2666e303091b None None] MFA Rules not processed for user `6e117d9bed4b41aebf89018e48e12dd7`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:50:42.075134 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-816f7185-3940-4be7-abaa-2666e303091b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:50:42.145322 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1186/2391] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:50:42 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:50:42.151057 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ad2684-fb5e-4adf-bd8a-cca3c7c172f5 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:42.217819 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ad2684-fb5e-4adf-bd8a-cca3c7c172f5 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:42.223450 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f703d895-0e3a-4232-9e94-f0cf0f9fd47b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:42.224061 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f703d895-0e3a-4232-9e94-f0cf0f9fd47b admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:42.224402 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f703d895-0e3a-4232-9e94-f0cf0f9fd47b admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:42.224738 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f703d895-0e3a-4232-9e94-f0cf0f9fd47b admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:42.225462 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f703d895-0e3a-4232-9e94-f0cf0f9fd47b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:42.227512 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f703d895-0e3a-4232-9e94-f0cf0f9fd47b admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:42.254216 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1206/2392] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jun 16 17:50:42 2022] POST /identity/v3/projects => generated 389 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:50:42.258186 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816f7185-3940-4be7-abaa-2666e303091b None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:42.269885 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816f7185-3940-4be7-abaa-2666e303091b None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:42.315745 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:50:42.328964 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c9c888-c09c-49b9-ba1f-fbeb1d8a1616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:42.329637 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c9c888-c09c-49b9-ba1f-fbeb1d8a1616 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:42.329924 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c9c888-c09c-49b9-ba1f-fbeb1d8a1616 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:42.330209 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c9c888-c09c-49b9-ba1f-fbeb1d8a1616 admin admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:42.331459 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c9c888-c09c-49b9-ba1f-fbeb1d8a1616 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:42.333541 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c9c888-c09c-49b9-ba1f-fbeb1d8a1616 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:42.353316 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1187/2393] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jun 16 17:50:42 2022] POST /identity/v3/users => generated 301 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:50:42.357316 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f703d895-0e3a-4232-9e94-f0cf0f9fd47b admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:42.368063 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f703d895-0e3a-4232-9e94-f0cf0f9fd47b admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:42.372807 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cdd03b-e7c6-4d43-8987-4e8f83c2e24c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:42.373463 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cdd03b-e7c6-4d43-8987-4e8f83c2e24c admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:42.373750 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cdd03b-e7c6-4d43-8987-4e8f83c2e24c admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:42.374032 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cdd03b-e7c6-4d43-8987-4e8f83c2e24c admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:42.374759 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cdd03b-e7c6-4d43-8987-4e8f83c2e24c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:42.376428 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cdd03b-e7c6-4d43-8987-4e8f83c2e24c admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:42.389135 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1207/2394] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:50:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:50:42.394002 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c9c888-c09c-49b9-ba1f-fbeb1d8a1616 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:42.405560 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c9c888-c09c-49b9-ba1f-fbeb1d8a1616 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:42.411491 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce6e159-effc-4d15-a508-cc82ca8d4fee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:42.412365 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce6e159-effc-4d15-a508-cc82ca8d4fee admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:42.412673 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce6e159-effc-4d15-a508-cc82ca8d4fee admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:42.412945 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce6e159-effc-4d15-a508-cc82ca8d4fee admin admin] PATH_INFO: `/v3/projects/df65e8645ff34149ac40a051a10ac014/users/0d24683624d4469abdffb5ee52b0ab2f/roles/3ca2dcbbd29d4a31a3ab78f2299bc239` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:42.427816 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce6e159-effc-4d15-a508-cc82ca8d4fee admin admin] RBAC: Authorizing `identity:create_grant(project_id=df65e8645ff34149ac40a051a10ac014, user_id=0d24683624d4469abdffb5ee52b0ab2f, role_id=3ca2dcbbd29d4a31a3ab78f2299bc239)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:42.430504 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce6e159-effc-4d15-a508-cc82ca8d4fee admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:42.444000 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1188/2395] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:50:42 2022] PUT /identity/v3/projects/df65e8645ff34149ac40a051a10ac014/users/0d24683624d4469abdffb5ee52b0ab2f/roles/3ca2dcbbd29d4a31a3ab78f2299bc239 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:50:42.448518 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cdd03b-e7c6-4d43-8987-4e8f83c2e24c admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:42.459809 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cdd03b-e7c6-4d43-8987-4e8f83c2e24c admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:42.503484 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:50:42.518186 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9dfc53-42f9-4498-bc6c-bbef4d4ce25a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:42.519517 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9dfc53-42f9-4498-bc6c-bbef4d4ce25a admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:42.519798 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9dfc53-42f9-4498-bc6c-bbef4d4ce25a admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:42.520076 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9dfc53-42f9-4498-bc6c-bbef4d4ce25a admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:42.520838 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9dfc53-42f9-4498-bc6c-bbef4d4ce25a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:42.522497 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9dfc53-42f9-4498-bc6c-bbef4d4ce25a admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:42.532216 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1208/2396] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:50:42 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 17:50:42.536398 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce6e159-effc-4d15-a508-cc82ca8d4fee admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:42.546624 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce6e159-effc-4d15-a508-cc82ca8d4fee admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:42.551371 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cf6001-3a44-41d0-99e7-247014ab4778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:42.552232 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cf6001-3a44-41d0-99e7-247014ab4778 admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:42.552557 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cf6001-3a44-41d0-99e7-247014ab4778 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:42.552837 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cf6001-3a44-41d0-99e7-247014ab4778 admin admin] PATH_INFO: `/v3/projects/df65e8645ff34149ac40a051a10ac014/users/0d24683624d4469abdffb5ee52b0ab2f/roles/3ca2dcbbd29d4a31a3ab78f2299bc239` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:42.554807 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cf6001-3a44-41d0-99e7-247014ab4778 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df65e8645ff34149ac40a051a10ac014, user_id=0d24683624d4469abdffb5ee52b0ab2f, role_id=3ca2dcbbd29d4a31a3ab78f2299bc239)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:42.557477 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cf6001-3a44-41d0-99e7-247014ab4778 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:42.567966 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1189/2397] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:50:42 2022] PUT /identity/v3/projects/df65e8645ff34149ac40a051a10ac014/users/0d24683624d4469abdffb5ee52b0ab2f/roles/3ca2dcbbd29d4a31a3ab78f2299bc239 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:50:42.577654 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdde554-12a4-4343-9d88-032bb61c65fd None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:42.577954 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdde554-12a4-4343-9d88-032bb61c65fd None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:42.578266 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdde554-12a4-4343-9d88-032bb61c65fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:42.606580 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-9fdde554-12a4-4343-9d88-032bb61c65fd None None] MFA Rules not processed for user `0d24683624d4469abdffb5ee52b0ab2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:50:42.607849 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-9fdde554-12a4-4343-9d88-032bb61c65fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:50:42.682944 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1209/2398] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:50:42 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:50:42.693941 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cf6001-3a44-41d0-99e7-247014ab4778 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:42.695770 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-76cf6001-3a44-41d0-99e7-247014ab4778 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:50:42.753909 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cf6001-3a44-41d0-99e7-247014ab4778 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:42.757620 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0780ca-1fc1-402b-96ac-f203829d6585 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24d5e2637b7043e08313909d5b06e329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:42.758268 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0780ca-1fc1-402b-96ac-f203829d6585 service cinder] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:42.758554 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0780ca-1fc1-402b-96ac-f203829d6585 service cinder] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:42.758833 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0780ca-1fc1-402b-96ac-f203829d6585 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:42.768210 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0780ca-1fc1-402b-96ac-f203829d6585 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:42.769756 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0780ca-1fc1-402b-96ac-f203829d6585 service cinder] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:42.779150 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1190/2399] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:50:42 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:50:42.914393 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdde554-12a4-4343-9d88-032bb61c65fd None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:42.977188 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdde554-12a4-4343-9d88-032bb61c65fd None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:42.981951 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf76a66-9d8c-4cba-a887-274579247605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:42.982820 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf76a66-9d8c-4cba-a887-274579247605 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:42.983102 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf76a66-9d8c-4cba-a887-274579247605 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:42.984935 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf76a66-9d8c-4cba-a887-274579247605 admin admin] PATH_INFO: `/v3/users/6e117d9bed4b41aebf89018e48e12dd7` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:42.984935 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf76a66-9d8c-4cba-a887-274579247605 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e117d9bed4b41aebf89018e48e12dd7)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:42.986856 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf76a66-9d8c-4cba-a887-274579247605 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:43.039516 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-adf76a66-9d8c-4cba-a887-274579247605 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e117d9bed4b41aebf89018e48e12dd7'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:43.052723 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-adf76a66-9d8c-4cba-a887-274579247605 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e117d9bed4b41aebf89018e48e12dd7'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:43.053411 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-adf76a66-9d8c-4cba-a887-274579247605 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e117d9bed4b41aebf89018e48e12dd7'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:43.060738 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:50:43.062860 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-adf76a66-9d8c-4cba-a887-274579247605 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e117d9bed4b41aebf89018e48e12dd7'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:43.073332 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-adf76a66-9d8c-4cba-a887-274579247605 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e117d9bed4b41aebf89018e48e12dd7'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:43.080361 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1210/2400] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jun 16 17:50:42 2022] DELETE /identity/v3/users/6e117d9bed4b41aebf89018e48e12dd7 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:50:43.085602 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64e85fe-ed85-4ae1-8dc5-416f4bc0fb2a None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:43.085895 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64e85fe-ed85-4ae1-8dc5-416f4bc0fb2a None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:43.086173 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64e85fe-ed85-4ae1-8dc5-416f4bc0fb2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:43.105840 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-b64e85fe-ed85-4ae1-8dc5-416f4bc0fb2a None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:50:43.107036 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-b64e85fe-ed85-4ae1-8dc5-416f4bc0fb2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:50:43.154484 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c682f2bd-a0fb-4917-943f-7bb9cf6fc114 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:43.155245 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c682f2bd-a0fb-4917-943f-7bb9cf6fc114 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:43.155543 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c682f2bd-a0fb-4917-943f-7bb9cf6fc114 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:43.178426 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-c682f2bd-a0fb-4917-943f-7bb9cf6fc114 None None] MFA Rules not processed for user `71443c30b4554719b5549493bf3cc00c`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:50:43.178859 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1191/2401] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:50:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:50:43.180122 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-c682f2bd-a0fb-4917-943f-7bb9cf6fc114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:50:43.192493 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64e85fe-ed85-4ae1-8dc5-416f4bc0fb2a None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:43.194002 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-b64e85fe-ed85-4ae1-8dc5-416f4bc0fb2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:50:43.254562 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1211/2402] 10.222.0.42 () {58 vars in 967 bytes} [Thu Jun 16 17:50:43 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:50:43.259193 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:50:43.276593 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64e85fe-ed85-4ae1-8dc5-416f4bc0fb2a None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:43.280575 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e93a4c-9232-4756-b580-7ac1f1ba22eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:43.281236 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e93a4c-9232-4756-b580-7ac1f1ba22eb service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:43.282433 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e93a4c-9232-4756-b580-7ac1f1ba22eb service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:43.282744 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e93a4c-9232-4756-b580-7ac1f1ba22eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:43.283193 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c682f2bd-a0fb-4917-943f-7bb9cf6fc114 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:43.285567 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-c682f2bd-a0fb-4917-943f-7bb9cf6fc114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:50:43.295979 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e93a4c-9232-4756-b580-7ac1f1ba22eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:43.297527 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e93a4c-9232-4756-b580-7ac1f1ba22eb service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:43.310603 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1192/2403] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:50:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 17:50:43.377301 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c682f2bd-a0fb-4917-943f-7bb9cf6fc114 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:43.384569 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b2b458-08e0-49e8-8cac-46ecf1ace42c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47b7a9cd615c41d6983a0007b475038c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:43.385563 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b2b458-08e0-49e8-8cac-46ecf1ace42c service swift] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:43.385898 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b2b458-08e0-49e8-8cac-46ecf1ace42c service swift] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:43.386536 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b2b458-08e0-49e8-8cac-46ecf1ace42c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:43.396674 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b2b458-08e0-49e8-8cac-46ecf1ace42c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:43.398422 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b2b458-08e0-49e8-8cac-46ecf1ace42c service swift] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:43.407394 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1212/2404] 10.222.0.42 () {60 vars in 1384 bytes} [Thu Jun 16 17:50:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 17:50:43.599180 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e93a4c-9232-4756-b580-7ac1f1ba22eb service neutron] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:43.600805 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-40e93a4c-9232-4756-b580-7ac1f1ba22eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:50:43.635144 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e93a4c-9232-4756-b580-7ac1f1ba22eb service neutron] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:43.640283 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac92e6c0-3207-4997-80c5-3ebac1cdd87a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:43.641234 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac92e6c0-3207-4997-80c5-3ebac1cdd87a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:43.641515 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac92e6c0-3207-4997-80c5-3ebac1cdd87a admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:43.641802 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac92e6c0-3207-4997-80c5-3ebac1cdd87a admin admin] PATH_INFO: `/v3/projects/c7460e48168d4a75a18b667ac5baf0a0` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:43.642958 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac92e6c0-3207-4997-80c5-3ebac1cdd87a admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7460e48168d4a75a18b667ac5baf0a0)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:43.645277 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac92e6c0-3207-4997-80c5-3ebac1cdd87a admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:43.697370 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-ac92e6c0-3207-4997-80c5-3ebac1cdd87a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7460e48168d4a75a18b667ac5baf0a0', 'request_id': 'req-ac92e6c0-3207-4997-80c5-3ebac1cdd87a'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:43.704633 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-ac92e6c0-3207-4997-80c5-3ebac1cdd87a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7460e48168d4a75a18b667ac5baf0a0', 'request_id': 'req-ac92e6c0-3207-4997-80c5-3ebac1cdd87a'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:43.710799 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-ac92e6c0-3207-4997-80c5-3ebac1cdd87a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7460e48168d4a75a18b667ac5baf0a0', 'request_id': 'req-ac92e6c0-3207-4997-80c5-3ebac1cdd87a'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:43.722568 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-ac92e6c0-3207-4997-80c5-3ebac1cdd87a admin admin] The token cache is being invalidate because project c7460e48168d4a75a18b667ac5baf0a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80357) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:50:43.722858 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-ac92e6c0-3207-4997-80c5-3ebac1cdd87a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:43.723510 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-ac92e6c0-3207-4997-80c5-3ebac1cdd87a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:43.725209 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1193/2405] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jun 16 17:50:43 2022] DELETE /identity/v3/projects/c7460e48168d4a75a18b667ac5baf0a0 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:50:43.729210 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b2b458-08e0-49e8-8cac-46ecf1ace42c service swift] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:43.740724 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b2b458-08e0-49e8-8cac-46ecf1ace42c service swift] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:43.800477 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee256205-5d5c-4df8-ad31-1710b0c4c256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:43.801834 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee256205-5d5c-4df8-ad31-1710b0c4c256 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:43.802360 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee256205-5d5c-4df8-ad31-1710b0c4c256 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:43.802866 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee256205-5d5c-4df8-ad31-1710b0c4c256 admin admin] PATH_INFO: `/v3/users/0d24683624d4469abdffb5ee52b0ab2f` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:43.804309 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee256205-5d5c-4df8-ad31-1710b0c4c256 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d24683624d4469abdffb5ee52b0ab2f)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:43.806956 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee256205-5d5c-4df8-ad31-1710b0c4c256 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:43.833435 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:50:43.858946 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ee256205-5d5c-4df8-ad31-1710b0c4c256 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d24683624d4469abdffb5ee52b0ab2f'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:43.873020 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ee256205-5d5c-4df8-ad31-1710b0c4c256 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d24683624d4469abdffb5ee52b0ab2f'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:43.873733 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ee256205-5d5c-4df8-ad31-1710b0c4c256 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d24683624d4469abdffb5ee52b0ab2f'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:43.881100 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ee256205-5d5c-4df8-ad31-1710b0c4c256 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d24683624d4469abdffb5ee52b0ab2f'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:43.889285 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ee256205-5d5c-4df8-ad31-1710b0c4c256 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d24683624d4469abdffb5ee52b0ab2f'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:43.895889 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1213/2406] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jun 16 17:50:43 2022] DELETE /identity/v3/users/0d24683624d4469abdffb5ee52b0ab2f => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:50:44.202411 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac92e6c0-3207-4997-80c5-3ebac1cdd87a admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:44.208297 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-ac92e6c0-3207-4997-80c5-3ebac1cdd87a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:50:44.225858 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac92e6c0-3207-4997-80c5-3ebac1cdd87a admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:44.307576 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f747213-00ad-4fd5-a7de-c835102704a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:44.308490 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f747213-00ad-4fd5-a7de-c835102704a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:44.308845 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f747213-00ad-4fd5-a7de-c835102704a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:44.309162 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f747213-00ad-4fd5-a7de-c835102704a0 admin admin] PATH_INFO: `/v3/projects/df65e8645ff34149ac40a051a10ac014` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:44.310668 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f747213-00ad-4fd5-a7de-c835102704a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=df65e8645ff34149ac40a051a10ac014)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:44.312455 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f747213-00ad-4fd5-a7de-c835102704a0 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:44.370985 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-6f747213-00ad-4fd5-a7de-c835102704a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df65e8645ff34149ac40a051a10ac014', 'request_id': 'req-6f747213-00ad-4fd5-a7de-c835102704a0'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:44.379982 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-6f747213-00ad-4fd5-a7de-c835102704a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df65e8645ff34149ac40a051a10ac014', 'request_id': 'req-6f747213-00ad-4fd5-a7de-c835102704a0'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:44.387679 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-6f747213-00ad-4fd5-a7de-c835102704a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df65e8645ff34149ac40a051a10ac014', 'request_id': 'req-6f747213-00ad-4fd5-a7de-c835102704a0'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:44.396579 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:50:44.403763 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-6f747213-00ad-4fd5-a7de-c835102704a0 admin admin] The token cache is being invalidate because project df65e8645ff34149ac40a051a10ac014 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80357) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:50:44.404308 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-6f747213-00ad-4fd5-a7de-c835102704a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:44.405800 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-6f747213-00ad-4fd5-a7de-c835102704a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:44.411697 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1194/2407] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jun 16 17:50:44 2022] DELETE /identity/v3/projects/df65e8645ff34149ac40a051a10ac014 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:50:44.413556 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48829674-57e8-46f5-ba5c-35b6af3fae1a None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:44.414009 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48829674-57e8-46f5-ba5c-35b6af3fae1a None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:44.414123 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48829674-57e8-46f5-ba5c-35b6af3fae1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:44.439725 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-48829674-57e8-46f5-ba5c-35b6af3fae1a None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:50:44.441727 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-48829674-57e8-46f5-ba5c-35b6af3fae1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:50:44.520055 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1214/2408] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:50:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:50:44.528097 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11550c6b-fc33-42dd-ae59-42ab2a00056e None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:44.528405 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11550c6b-fc33-42dd-ae59-42ab2a00056e None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:44.528719 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11550c6b-fc33-42dd-ae59-42ab2a00056e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:44.553187 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-11550c6b-fc33-42dd-ae59-42ab2a00056e None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:50:44.554473 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-11550c6b-fc33-42dd-ae59-42ab2a00056e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:50:44.567578 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1195/2409] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:50:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:50:44.570393 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48829674-57e8-46f5-ba5c-35b6af3fae1a None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:44.582842 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48829674-57e8-46f5-ba5c-35b6af3fae1a None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:44.590307 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e743aab4-f4e5-4363-bbc2-9ff8bef588ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:44.590988 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e743aab4-f4e5-4363-bbc2-9ff8bef588ce admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:44.591282 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e743aab4-f4e5-4363-bbc2-9ff8bef588ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:44.591560 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e743aab4-f4e5-4363-bbc2-9ff8bef588ce admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:44.592290 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e743aab4-f4e5-4363-bbc2-9ff8bef588ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:50:44.592782 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e743aab4-f4e5-4363-bbc2-9ff8bef588ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:44.594634 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e743aab4-f4e5-4363-bbc2-9ff8bef588ce admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:44.611286 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1215/2410] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jun 16 17:50:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:50:44.616641 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11550c6b-fc33-42dd-ae59-42ab2a00056e None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:44.630097 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11550c6b-fc33-42dd-ae59-42ab2a00056e None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:44.635375 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e534366e-8b1d-42be-a3dd-13db4c0d5d7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:44.636284 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e534366e-8b1d-42be-a3dd-13db4c0d5d7f admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:44.636608 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e534366e-8b1d-42be-a3dd-13db4c0d5d7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:44.636886 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e534366e-8b1d-42be-a3dd-13db4c0d5d7f admin admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:44.637576 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e534366e-8b1d-42be-a3dd-13db4c0d5d7f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:44.640173 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e534366e-8b1d-42be-a3dd-13db4c0d5d7f admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:44.671910 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1196/2411] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jun 16 17:50:44 2022] POST /identity/v3/projects => generated 379 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:50:44.677694 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e743aab4-f4e5-4363-bbc2-9ff8bef588ce admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:44.692542 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e743aab4-f4e5-4363-bbc2-9ff8bef588ce admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:44.758847 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ef1b3f-ed57-4ef5-9633-2406499dc9df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:44.759578 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ef1b3f-ed57-4ef5-9633-2406499dc9df admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:44.759802 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ef1b3f-ed57-4ef5-9633-2406499dc9df admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:44.760081 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ef1b3f-ed57-4ef5-9633-2406499dc9df admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:44.761754 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ef1b3f-ed57-4ef5-9633-2406499dc9df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:44.764550 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ef1b3f-ed57-4ef5-9633-2406499dc9df admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:44.790038 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1216/2412] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jun 16 17:50:44 2022] POST /identity/v3/users => generated 290 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:50:44.794409 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e534366e-8b1d-42be-a3dd-13db4c0d5d7f admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:44.807246 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e534366e-8b1d-42be-a3dd-13db4c0d5d7f admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:44.812827 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab274d8-ddeb-43b8-99b9-227e820aa10a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:44.813619 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab274d8-ddeb-43b8-99b9-227e820aa10a admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:44.813949 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab274d8-ddeb-43b8-99b9-227e820aa10a admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:44.814273 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab274d8-ddeb-43b8-99b9-227e820aa10a admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:44.815483 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab274d8-ddeb-43b8-99b9-227e820aa10a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:44.817685 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab274d8-ddeb-43b8-99b9-227e820aa10a admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:44.828900 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1197/2413] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:50:44 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:50:44.833664 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ef1b3f-ed57-4ef5-9633-2406499dc9df admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:44.845813 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ef1b3f-ed57-4ef5-9633-2406499dc9df admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:44.851184 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69374178-60fe-4710-9f1f-18e2220cf717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:44.851850 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69374178-60fe-4710-9f1f-18e2220cf717 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:44.852157 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69374178-60fe-4710-9f1f-18e2220cf717 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:44.852471 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69374178-60fe-4710-9f1f-18e2220cf717 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:44.853199 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69374178-60fe-4710-9f1f-18e2220cf717 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:44.854941 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69374178-60fe-4710-9f1f-18e2220cf717 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:44.870877 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1217/2414] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:50:44 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:50:44.877227 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab274d8-ddeb-43b8-99b9-227e820aa10a admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:44.891930 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab274d8-ddeb-43b8-99b9-227e820aa10a admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:44.897941 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33519cd-5498-4a6c-be3c-f923bb6b6549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:44.898832 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33519cd-5498-4a6c-be3c-f923bb6b6549 admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:44.899112 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33519cd-5498-4a6c-be3c-f923bb6b6549 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:44.899386 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33519cd-5498-4a6c-be3c-f923bb6b6549 admin admin] PATH_INFO: `/v3/projects/b00f3d741e4b46649a32cde087bb5bc0/users/9fc9ffd8f55048bc9d18a0805c988668/roles/0be2075946f54c8485f1c275f746478a` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:44.920201 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33519cd-5498-4a6c-be3c-f923bb6b6549 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b00f3d741e4b46649a32cde087bb5bc0, user_id=9fc9ffd8f55048bc9d18a0805c988668, role_id=0be2075946f54c8485f1c275f746478a)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:44.926001 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33519cd-5498-4a6c-be3c-f923bb6b6549 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:44.940889 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1198/2415] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:50:44 2022] PUT /identity/v3/projects/b00f3d741e4b46649a32cde087bb5bc0/users/9fc9ffd8f55048bc9d18a0805c988668/roles/0be2075946f54c8485f1c275f746478a => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:50:44.953467 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb460d1-7f4f-442b-80f0-710076dacc71 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:44.953800 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb460d1-7f4f-442b-80f0-710076dacc71 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:44.954191 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb460d1-7f4f-442b-80f0-710076dacc71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:44.967441 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:50:44.983345 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-bbb460d1-7f4f-442b-80f0-710076dacc71 None None] MFA Rules not processed for user `9fc9ffd8f55048bc9d18a0805c988668`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:50:44.984646 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-bbb460d1-7f4f-442b-80f0-710076dacc71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:50:45.049300 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1218/2416] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:50:44 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:50:45.054428 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33519cd-5498-4a6c-be3c-f923bb6b6549 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:45.119723 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33519cd-5498-4a6c-be3c-f923bb6b6549 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:45.124510 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ed83ff-7de5-4798-810c-d2774c11e6b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:45.125173 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ed83ff-7de5-4798-810c-d2774c11e6b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:45.125459 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ed83ff-7de5-4798-810c-d2774c11e6b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:45.125767 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ed83ff-7de5-4798-810c-d2774c11e6b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:45.126466 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ed83ff-7de5-4798-810c-d2774c11e6b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:45.128946 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ed83ff-7de5-4798-810c-d2774c11e6b7 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:45.155080 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1199/2417] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jun 16 17:50:45 2022] POST /identity/v3/projects => generated 379 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:50:45.159831 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb460d1-7f4f-442b-80f0-710076dacc71 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:45.170575 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb460d1-7f4f-442b-80f0-710076dacc71 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:45.226157 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d18a4a-64e2-4456-b5f0-83c80754cfdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:45.226854 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d18a4a-64e2-4456-b5f0-83c80754cfdf admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:45.227131 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d18a4a-64e2-4456-b5f0-83c80754cfdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:45.227402 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d18a4a-64e2-4456-b5f0-83c80754cfdf admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:45.228096 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d18a4a-64e2-4456-b5f0-83c80754cfdf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:45.230024 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d18a4a-64e2-4456-b5f0-83c80754cfdf admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:45.249850 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1219/2418] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jun 16 17:50:45 2022] POST /identity/v3/users => generated 290 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:50:45.254336 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ed83ff-7de5-4798-810c-d2774c11e6b7 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:45.264896 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ed83ff-7de5-4798-810c-d2774c11e6b7 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:45.270300 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff038936-8162-4b13-823b-a1562ab5e8bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:45.270963 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff038936-8162-4b13-823b-a1562ab5e8bc admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:45.271246 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff038936-8162-4b13-823b-a1562ab5e8bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:45.271553 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff038936-8162-4b13-823b-a1562ab5e8bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:45.272277 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff038936-8162-4b13-823b-a1562ab5e8bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:45.273936 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff038936-8162-4b13-823b-a1562ab5e8bc admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:45.285395 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1200/2419] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:50:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:50:45.289491 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d18a4a-64e2-4456-b5f0-83c80754cfdf admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:45.303088 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d18a4a-64e2-4456-b5f0-83c80754cfdf admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:45.309084 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cc3d0b-2b23-4cba-9cf3-c9129cc1b688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:45.309746 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cc3d0b-2b23-4cba-9cf3-c9129cc1b688 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:45.310030 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cc3d0b-2b23-4cba-9cf3-c9129cc1b688 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:45.310325 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cc3d0b-2b23-4cba-9cf3-c9129cc1b688 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:45.311073 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cc3d0b-2b23-4cba-9cf3-c9129cc1b688 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:45.312744 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cc3d0b-2b23-4cba-9cf3-c9129cc1b688 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:45.328323 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1220/2420] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:50:45 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:50:45.332857 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff038936-8162-4b13-823b-a1562ab5e8bc admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:45.343006 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff038936-8162-4b13-823b-a1562ab5e8bc admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:45.348830 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114158f8-5dba-4be5-a468-ccda0ebe66c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:45.349794 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114158f8-5dba-4be5-a468-ccda0ebe66c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:45.349983 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114158f8-5dba-4be5-a468-ccda0ebe66c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:45.350264 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114158f8-5dba-4be5-a468-ccda0ebe66c8 admin admin] PATH_INFO: `/v3/projects/12ae024a6c36462e8ede739817cee650/users/a9616079dbf44222bf5a19232229cf97/roles/0be2075946f54c8485f1c275f746478a` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:45.365171 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114158f8-5dba-4be5-a468-ccda0ebe66c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=12ae024a6c36462e8ede739817cee650, user_id=a9616079dbf44222bf5a19232229cf97, role_id=0be2075946f54c8485f1c275f746478a)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:45.367890 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114158f8-5dba-4be5-a468-ccda0ebe66c8 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:45.381476 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1201/2421] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:50:45 2022] PUT /identity/v3/projects/12ae024a6c36462e8ede739817cee650/users/a9616079dbf44222bf5a19232229cf97/roles/0be2075946f54c8485f1c275f746478a => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:50:45.392737 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387aafd6-c31a-410b-aa67-679bf3881396 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:45.393059 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387aafd6-c31a-410b-aa67-679bf3881396 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:45.393961 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387aafd6-c31a-410b-aa67-679bf3881396 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:45.420158 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-387aafd6-c31a-410b-aa67-679bf3881396 None None] MFA Rules not processed for user `a9616079dbf44222bf5a19232229cf97`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:50:45.421499 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-387aafd6-c31a-410b-aa67-679bf3881396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:50:45.488065 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1221/2422] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:50:45 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:50:45.492564 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114158f8-5dba-4be5-a468-ccda0ebe66c8 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:45.542653 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:50:45.558198 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114158f8-5dba-4be5-a468-ccda0ebe66c8 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:45.562886 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb6e040-d0eb-4f32-8ec6-132f5c3b85b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:45.563609 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb6e040-d0eb-4f32-8ec6-132f5c3b85b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:45.563919 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb6e040-d0eb-4f32-8ec6-132f5c3b85b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:45.564234 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb6e040-d0eb-4f32-8ec6-132f5c3b85b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:45.564984 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb6e040-d0eb-4f32-8ec6-132f5c3b85b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:45.567582 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb6e040-d0eb-4f32-8ec6-132f5c3b85b7 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:45.593184 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1202/2423] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jun 16 17:50:45 2022] POST /identity/v3/projects => generated 379 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:50:45.597651 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387aafd6-c31a-410b-aa67-679bf3881396 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:45.604146 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:50:45.608584 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387aafd6-c31a-410b-aa67-679bf3881396 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:45.667556 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b55e77-be27-4a6d-88e5-b11f70ef4110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:45.668222 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b55e77-be27-4a6d-88e5-b11f70ef4110 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:45.668546 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b55e77-be27-4a6d-88e5-b11f70ef4110 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:45.668824 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b55e77-be27-4a6d-88e5-b11f70ef4110 admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:45.669504 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b55e77-be27-4a6d-88e5-b11f70ef4110 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:45.671407 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b55e77-be27-4a6d-88e5-b11f70ef4110 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:45.693232 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1222/2424] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jun 16 17:50:45 2022] POST /identity/v3/users => generated 296 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:50:45.696858 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb6e040-d0eb-4f32-8ec6-132f5c3b85b7 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:45.707388 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb6e040-d0eb-4f32-8ec6-132f5c3b85b7 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:45.714742 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbde964a-9cdd-4e56-8a52-44f5da0d5673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:45.715487 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbde964a-9cdd-4e56-8a52-44f5da0d5673 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:45.715800 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbde964a-9cdd-4e56-8a52-44f5da0d5673 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:45.716117 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbde964a-9cdd-4e56-8a52-44f5da0d5673 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:45.716984 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbde964a-9cdd-4e56-8a52-44f5da0d5673 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:45.718903 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbde964a-9cdd-4e56-8a52-44f5da0d5673 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:45.730960 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1203/2425] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:50:45 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:50:45.735012 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b55e77-be27-4a6d-88e5-b11f70ef4110 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:45.745857 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b55e77-be27-4a6d-88e5-b11f70ef4110 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:45.750502 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62c67c3-d69a-4a3e-8f21-34a0bb3bfdc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:45.751378 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62c67c3-d69a-4a3e-8f21-34a0bb3bfdc0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:45.751658 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62c67c3-d69a-4a3e-8f21-34a0bb3bfdc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:45.751932 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62c67c3-d69a-4a3e-8f21-34a0bb3bfdc0 admin admin] PATH_INFO: `/v3/projects/e4ba1f116cbf47abab03ecc1e6b6cc69/users/bb9169345c804ebe828604966f24c805/roles/3ca2dcbbd29d4a31a3ab78f2299bc239` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:45.767997 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62c67c3-d69a-4a3e-8f21-34a0bb3bfdc0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4ba1f116cbf47abab03ecc1e6b6cc69, user_id=bb9169345c804ebe828604966f24c805, role_id=3ca2dcbbd29d4a31a3ab78f2299bc239)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:45.770865 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62c67c3-d69a-4a3e-8f21-34a0bb3bfdc0 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:45.784691 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1223/2426] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:50:45 2022] PUT /identity/v3/projects/e4ba1f116cbf47abab03ecc1e6b6cc69/users/bb9169345c804ebe828604966f24c805/roles/3ca2dcbbd29d4a31a3ab78f2299bc239 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:50:45.795025 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbde964a-9cdd-4e56-8a52-44f5da0d5673 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:45.806396 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbde964a-9cdd-4e56-8a52-44f5da0d5673 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:45.867499 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a587e3-c950-4966-b865-2a8833a8c59c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:45.868171 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a587e3-c950-4966-b865-2a8833a8c59c admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:45.868487 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a587e3-c950-4966-b865-2a8833a8c59c admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:45.868766 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a587e3-c950-4966-b865-2a8833a8c59c admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:45.869494 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a587e3-c950-4966-b865-2a8833a8c59c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:45.871096 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a587e3-c950-4966-b865-2a8833a8c59c admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:45.881579 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1204/2427] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:50:45 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:50:45.885761 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62c67c3-d69a-4a3e-8f21-34a0bb3bfdc0 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:45.896852 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62c67c3-d69a-4a3e-8f21-34a0bb3bfdc0 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:45.901372 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480bf3a5-3e10-4ab5-88a4-b5dd2521bc90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:45.902244 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480bf3a5-3e10-4ab5-88a4-b5dd2521bc90 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:45.902519 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480bf3a5-3e10-4ab5-88a4-b5dd2521bc90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:45.902789 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480bf3a5-3e10-4ab5-88a4-b5dd2521bc90 admin admin] PATH_INFO: `/v3/projects/e4ba1f116cbf47abab03ecc1e6b6cc69/users/bb9169345c804ebe828604966f24c805/roles/3ca2dcbbd29d4a31a3ab78f2299bc239` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:45.904777 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480bf3a5-3e10-4ab5-88a4-b5dd2521bc90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4ba1f116cbf47abab03ecc1e6b6cc69, user_id=bb9169345c804ebe828604966f24c805, role_id=3ca2dcbbd29d4a31a3ab78f2299bc239)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:45.907377 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480bf3a5-3e10-4ab5-88a4-b5dd2521bc90 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:45.917936 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1224/2428] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:50:45 2022] PUT /identity/v3/projects/e4ba1f116cbf47abab03ecc1e6b6cc69/users/bb9169345c804ebe828604966f24c805/roles/3ca2dcbbd29d4a31a3ab78f2299bc239 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:50:45.929515 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93303d97-30fa-4438-a0e7-d712d3160181 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:45.929807 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93303d97-30fa-4438-a0e7-d712d3160181 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:45.930087 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93303d97-30fa-4438-a0e7-d712d3160181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:45.956426 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-93303d97-30fa-4438-a0e7-d712d3160181 None None] MFA Rules not processed for user `bb9169345c804ebe828604966f24c805`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:50:45.958563 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-93303d97-30fa-4438-a0e7-d712d3160181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:50:46.031806 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1205/2429] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:50:45 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:50:46.043281 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480bf3a5-3e10-4ab5-88a4-b5dd2521bc90 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:46.045083 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-480bf3a5-3e10-4ab5-88a4-b5dd2521bc90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:50:46.104987 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480bf3a5-3e10-4ab5-88a4-b5dd2521bc90 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:46.108617 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f7cf47-ba18-4e03-abbc-558c1e38d60a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24d5e2637b7043e08313909d5b06e329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:46.109285 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f7cf47-ba18-4e03-abbc-558c1e38d60a service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:46.109562 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f7cf47-ba18-4e03-abbc-558c1e38d60a service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:46.109832 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f7cf47-ba18-4e03-abbc-558c1e38d60a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:46.119491 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f7cf47-ba18-4e03-abbc-558c1e38d60a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:46.121046 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f7cf47-ba18-4e03-abbc-558c1e38d60a service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:46.131223 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1225/2430] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:50:46 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:50:46.173943 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93303d97-30fa-4438-a0e7-d712d3160181 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:46.175749 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-93303d97-30fa-4438-a0e7-d712d3160181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:50:46.191609 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93303d97-30fa-4438-a0e7-d712d3160181 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:46.196012 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91f6ae0-110c-4b9d-9752-f4ba3b24bfe3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24d5e2637b7043e08313909d5b06e329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:46.196857 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91f6ae0-110c-4b9d-9752-f4ba3b24bfe3 service cinder] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:46.197143 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91f6ae0-110c-4b9d-9752-f4ba3b24bfe3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:46.197422 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91f6ae0-110c-4b9d-9752-f4ba3b24bfe3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:46.244352 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:50:46.265055 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91f6ae0-110c-4b9d-9752-f4ba3b24bfe3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:46.266541 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91f6ae0-110c-4b9d-9752-f4ba3b24bfe3 service cinder] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:46.275737 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1206/2431] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:50:46 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:50:49.167486 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f7cf47-ba18-4e03-abbc-558c1e38d60a service cinder] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:49.231984 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f7cf47-ba18-4e03-abbc-558c1e38d60a service cinder] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:49.236642 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e4d6a7-d7d3-4d49-9d5b-2c0416e87942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:49.237513 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e4d6a7-d7d3-4d49-9d5b-2c0416e87942 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:49.237793 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e4d6a7-d7d3-4d49-9d5b-2c0416e87942 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:49.238067 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e4d6a7-d7d3-4d49-9d5b-2c0416e87942 admin admin] PATH_INFO: `/v3/users/9fc9ffd8f55048bc9d18a0805c988668` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:49.240190 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e4d6a7-d7d3-4d49-9d5b-2c0416e87942 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9fc9ffd8f55048bc9d18a0805c988668)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:49.242144 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e4d6a7-d7d3-4d49-9d5b-2c0416e87942 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:49.243893 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:50:49.295266 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-e9e4d6a7-d7d3-4d49-9d5b-2c0416e87942 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9fc9ffd8f55048bc9d18a0805c988668'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:49.308977 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-e9e4d6a7-d7d3-4d49-9d5b-2c0416e87942 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9fc9ffd8f55048bc9d18a0805c988668'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:49.309700 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-e9e4d6a7-d7d3-4d49-9d5b-2c0416e87942 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9fc9ffd8f55048bc9d18a0805c988668'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:49.316544 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-e9e4d6a7-d7d3-4d49-9d5b-2c0416e87942 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9fc9ffd8f55048bc9d18a0805c988668'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:49.324027 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-e9e4d6a7-d7d3-4d49-9d5b-2c0416e87942 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9fc9ffd8f55048bc9d18a0805c988668'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:49.330107 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1226/2432] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jun 16 17:50:49 2022] DELETE /identity/v3/users/9fc9ffd8f55048bc9d18a0805c988668 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:50:49.335268 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34ce32e-2762-4a89-9d9b-ea22ec89afd0 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:49.335736 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34ce32e-2762-4a89-9d9b-ea22ec89afd0 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:49.335874 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34ce32e-2762-4a89-9d9b-ea22ec89afd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:49.354152 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-a34ce32e-2762-4a89-9d9b-ea22ec89afd0 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:50:49.355376 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-a34ce32e-2762-4a89-9d9b-ea22ec89afd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:50:49.419056 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1207/2433] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:50:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:50:49.432386 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e4d6a7-d7d3-4d49-9d5b-2c0416e87942 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:49.433850 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-e9e4d6a7-d7d3-4d49-9d5b-2c0416e87942 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:50:49.468335 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34ce32e-2762-4a89-9d9b-ea22ec89afd0 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:49.470047 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-a34ce32e-2762-4a89-9d9b-ea22ec89afd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:50:49.483623 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34ce32e-2762-4a89-9d9b-ea22ec89afd0 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:49.488468 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d76e66-5d2f-41ca-ae39-bc9534665841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:49.489436 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d76e66-5d2f-41ca-ae39-bc9534665841 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:49.489678 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d76e66-5d2f-41ca-ae39-bc9534665841 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:49.489959 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d76e66-5d2f-41ca-ae39-bc9534665841 admin admin] PATH_INFO: `/v3/users/0b50a0e55730403cb3246424988d3819` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:49.491281 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d76e66-5d2f-41ca-ae39-bc9534665841 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b50a0e55730403cb3246424988d3819)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:49.493260 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d76e66-5d2f-41ca-ae39-bc9534665841 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:49.498491 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e4d6a7-d7d3-4d49-9d5b-2c0416e87942 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:49.502590 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c068dcec-bfda-4693-b6a6-d45591c538e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:49.503245 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c068dcec-bfda-4693-b6a6-d45591c538e9 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:49.503532 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c068dcec-bfda-4693-b6a6-d45591c538e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:49.503808 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c068dcec-bfda-4693-b6a6-d45591c538e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:49.515148 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c068dcec-bfda-4693-b6a6-d45591c538e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:49.516823 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c068dcec-bfda-4693-b6a6-d45591c538e9 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:49.529058 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1227/2434] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:50:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:50:49.543883 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-d3d76e66-5d2f-41ca-ae39-bc9534665841 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b50a0e55730403cb3246424988d3819'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:49.544666 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-d3d76e66-5d2f-41ca-ae39-bc9534665841 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b50a0e55730403cb3246424988d3819'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:49.553672 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-d3d76e66-5d2f-41ca-ae39-bc9534665841 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b50a0e55730403cb3246424988d3819'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:49.554530 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-d3d76e66-5d2f-41ca-ae39-bc9534665841 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b50a0e55730403cb3246424988d3819'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:49.569084 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-d3d76e66-5d2f-41ca-ae39-bc9534665841 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b50a0e55730403cb3246424988d3819'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:49.582886 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1208/2435] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jun 16 17:50:49 2022] DELETE /identity/v3/users/0b50a0e55730403cb3246424988d3819 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:50:49.587437 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dbf84c-b200-4ef4-b4a5-947dac0e96db None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:49.587804 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dbf84c-b200-4ef4-b4a5-947dac0e96db None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:49.588169 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dbf84c-b200-4ef4-b4a5-947dac0e96db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:49.607946 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-a2dbf84c-b200-4ef4-b4a5-947dac0e96db None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:50:49.609191 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-a2dbf84c-b200-4ef4-b4a5-947dac0e96db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:50:49.702136 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1228/2436] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:50:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:50:49.714184 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d76e66-5d2f-41ca-ae39-bc9534665841 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:49.715617 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-d3d76e66-5d2f-41ca-ae39-bc9534665841 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:50:49.775674 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d76e66-5d2f-41ca-ae39-bc9534665841 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:49.779389 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce18d44-cd45-4dc8-b474-29a9d67634a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:49.780211 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce18d44-cd45-4dc8-b474-29a9d67634a5 service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:49.780525 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce18d44-cd45-4dc8-b474-29a9d67634a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:49.780798 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce18d44-cd45-4dc8-b474-29a9d67634a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:49.791171 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:50:49.793871 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce18d44-cd45-4dc8-b474-29a9d67634a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:49.795731 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce18d44-cd45-4dc8-b474-29a9d67634a5 service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:49.807868 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1209/2437] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:50:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:50:49.833673 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dbf84c-b200-4ef4-b4a5-947dac0e96db None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:49.835788 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-a2dbf84c-b200-4ef4-b4a5-947dac0e96db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:50:49.851863 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dbf84c-b200-4ef4-b4a5-947dac0e96db None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:49.866766 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20957c75-6b52-4824-a9c6-174016d9b68b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:49.866766 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20957c75-6b52-4824-a9c6-174016d9b68b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:49.866766 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20957c75-6b52-4824-a9c6-174016d9b68b admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:49.866766 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20957c75-6b52-4824-a9c6-174016d9b68b admin admin] PATH_INFO: `/v3/projects/b00f3d741e4b46649a32cde087bb5bc0` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:49.876041 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20957c75-6b52-4824-a9c6-174016d9b68b admin admin] RBAC: Authorizing `identity:delete_project(project_id=b00f3d741e4b46649a32cde087bb5bc0)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:49.879153 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20957c75-6b52-4824-a9c6-174016d9b68b admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:49.903865 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:50:49.941006 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-20957c75-6b52-4824-a9c6-174016d9b68b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b00f3d741e4b46649a32cde087bb5bc0', 'request_id': 'req-20957c75-6b52-4824-a9c6-174016d9b68b'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:49.948670 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-20957c75-6b52-4824-a9c6-174016d9b68b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b00f3d741e4b46649a32cde087bb5bc0', 'request_id': 'req-20957c75-6b52-4824-a9c6-174016d9b68b'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:49.955039 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-20957c75-6b52-4824-a9c6-174016d9b68b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b00f3d741e4b46649a32cde087bb5bc0', 'request_id': 'req-20957c75-6b52-4824-a9c6-174016d9b68b'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:49.965941 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-20957c75-6b52-4824-a9c6-174016d9b68b admin admin] The token cache is being invalidate because project b00f3d741e4b46649a32cde087bb5bc0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:50:49.966233 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-20957c75-6b52-4824-a9c6-174016d9b68b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:49.966906 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-20957c75-6b52-4824-a9c6-174016d9b68b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:49.971334 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1229/2438] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jun 16 17:50:49 2022] DELETE /identity/v3/projects/b00f3d741e4b46649a32cde087bb5bc0 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:50:49.972526 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce18d44-cd45-4dc8-b474-29a9d67634a5 service neutron] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:49.984039 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce18d44-cd45-4dc8-b474-29a9d67634a5 service neutron] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:50.049886 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3daa18-f08c-4fd2-96ef-fecb39cdfaa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:50.050753 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3daa18-f08c-4fd2-96ef-fecb39cdfaa3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:50.051033 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3daa18-f08c-4fd2-96ef-fecb39cdfaa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:50.051973 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3daa18-f08c-4fd2-96ef-fecb39cdfaa3 admin admin] PATH_INFO: `/v3/users/a9616079dbf44222bf5a19232229cf97` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:50.053798 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3daa18-f08c-4fd2-96ef-fecb39cdfaa3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9616079dbf44222bf5a19232229cf97)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:50.055742 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3daa18-f08c-4fd2-96ef-fecb39cdfaa3 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:50.104294 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-5b3daa18-f08c-4fd2-96ef-fecb39cdfaa3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9616079dbf44222bf5a19232229cf97'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:50.105334 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-5b3daa18-f08c-4fd2-96ef-fecb39cdfaa3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9616079dbf44222bf5a19232229cf97'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:50.113079 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20957c75-6b52-4824-a9c6-174016d9b68b admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:50.113211 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-5b3daa18-f08c-4fd2-96ef-fecb39cdfaa3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9616079dbf44222bf5a19232229cf97'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:50.114652 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-5b3daa18-f08c-4fd2-96ef-fecb39cdfaa3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9616079dbf44222bf5a19232229cf97'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:50.115223 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-20957c75-6b52-4824-a9c6-174016d9b68b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:50:50.132015 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20957c75-6b52-4824-a9c6-174016d9b68b admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:50.133076 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-5b3daa18-f08c-4fd2-96ef-fecb39cdfaa3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9616079dbf44222bf5a19232229cf97'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:50.137428 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2845da5d-183c-451c-aaba-90cd18336dac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:50.138549 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2845da5d-183c-451c-aaba-90cd18336dac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:50.138837 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2845da5d-183c-451c-aaba-90cd18336dac admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:50.139112 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2845da5d-183c-451c-aaba-90cd18336dac admin admin] PATH_INFO: `/v3/projects/2411977f39484513adcaf653cd25ab58` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:50.140341 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2845da5d-183c-451c-aaba-90cd18336dac admin admin] RBAC: Authorizing `identity:delete_project(project_id=2411977f39484513adcaf653cd25ab58)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:50.142383 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2845da5d-183c-451c-aaba-90cd18336dac admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:50.151598 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1210/2439] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jun 16 17:50:49 2022] DELETE /identity/v3/users/a9616079dbf44222bf5a19232229cf97 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:50:50.203275 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-2845da5d-183c-451c-aaba-90cd18336dac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2411977f39484513adcaf653cd25ab58', 'request_id': 'req-2845da5d-183c-451c-aaba-90cd18336dac'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:50.212225 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-2845da5d-183c-451c-aaba-90cd18336dac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2411977f39484513adcaf653cd25ab58', 'request_id': 'req-2845da5d-183c-451c-aaba-90cd18336dac'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:50.219332 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-2845da5d-183c-451c-aaba-90cd18336dac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2411977f39484513adcaf653cd25ab58', 'request_id': 'req-2845da5d-183c-451c-aaba-90cd18336dac'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:50.231170 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-2845da5d-183c-451c-aaba-90cd18336dac admin admin] The token cache is being invalidate because project 2411977f39484513adcaf653cd25ab58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:50:50.231455 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-2845da5d-183c-451c-aaba-90cd18336dac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:50.232282 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-2845da5d-183c-451c-aaba-90cd18336dac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:50.234076 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1230/2440] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jun 16 17:50:50 2022] DELETE /identity/v3/projects/2411977f39484513adcaf653cd25ab58 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:50:50.238311 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3daa18-f08c-4fd2-96ef-fecb39cdfaa3 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:50.239976 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-5b3daa18-f08c-4fd2-96ef-fecb39cdfaa3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:50:50.313592 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3daa18-f08c-4fd2-96ef-fecb39cdfaa3 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:50.335587 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:50:50.377080 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3bc9db-941c-4782-aa7e-e7df2bac4954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:50.377960 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3bc9db-941c-4782-aa7e-e7df2bac4954 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:50.378283 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3bc9db-941c-4782-aa7e-e7df2bac4954 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:50.378580 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3bc9db-941c-4782-aa7e-e7df2bac4954 admin admin] PATH_INFO: `/v3/users/0ea050c26a554c00b2c92ef57caffa7c` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:50.380170 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3bc9db-941c-4782-aa7e-e7df2bac4954 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ea050c26a554c00b2c92ef57caffa7c)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:50.382141 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3bc9db-941c-4782-aa7e-e7df2bac4954 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:50.431887 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-be3bc9db-941c-4782-aa7e-e7df2bac4954 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ea050c26a554c00b2c92ef57caffa7c'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:50.432596 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-be3bc9db-941c-4782-aa7e-e7df2bac4954 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ea050c26a554c00b2c92ef57caffa7c'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:50.441543 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-be3bc9db-941c-4782-aa7e-e7df2bac4954 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ea050c26a554c00b2c92ef57caffa7c'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:50.442414 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-be3bc9db-941c-4782-aa7e-e7df2bac4954 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ea050c26a554c00b2c92ef57caffa7c'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:50.447627 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2845da5d-183c-451c-aaba-90cd18336dac admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:50.449279 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-2845da5d-183c-451c-aaba-90cd18336dac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:50:50.461839 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-be3bc9db-941c-4782-aa7e-e7df2bac4954 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ea050c26a554c00b2c92ef57caffa7c'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:50.470646 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2845da5d-183c-451c-aaba-90cd18336dac admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:50.476118 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1921825d-9250-421d-83c4-bf0c2b2d1504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:50.476655 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1211/2441] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jun 16 17:50:50 2022] DELETE /identity/v3/users/0ea050c26a554c00b2c92ef57caffa7c => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:50:50.477015 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1921825d-9250-421d-83c4-bf0c2b2d1504 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:50.477432 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1921825d-9250-421d-83c4-bf0c2b2d1504 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:50.477745 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1921825d-9250-421d-83c4-bf0c2b2d1504 admin admin] PATH_INFO: `/v3/projects/12ae024a6c36462e8ede739817cee650` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:50.478943 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1921825d-9250-421d-83c4-bf0c2b2d1504 admin admin] RBAC: Authorizing `identity:delete_project(project_id=12ae024a6c36462e8ede739817cee650)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:50.480956 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1921825d-9250-421d-83c4-bf0c2b2d1504 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:50.533895 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-1921825d-9250-421d-83c4-bf0c2b2d1504 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12ae024a6c36462e8ede739817cee650', 'request_id': 'req-1921825d-9250-421d-83c4-bf0c2b2d1504'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:50.542361 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-1921825d-9250-421d-83c4-bf0c2b2d1504 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12ae024a6c36462e8ede739817cee650', 'request_id': 'req-1921825d-9250-421d-83c4-bf0c2b2d1504'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:50.549683 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-1921825d-9250-421d-83c4-bf0c2b2d1504 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12ae024a6c36462e8ede739817cee650', 'request_id': 'req-1921825d-9250-421d-83c4-bf0c2b2d1504'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:50.560782 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-1921825d-9250-421d-83c4-bf0c2b2d1504 admin admin] The token cache is being invalidate because project 12ae024a6c36462e8ede739817cee650 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:50:50.561159 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-1921825d-9250-421d-83c4-bf0c2b2d1504 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:50.561899 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-1921825d-9250-421d-83c4-bf0c2b2d1504 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:50.563522 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1231/2442] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jun 16 17:50:50 2022] DELETE /identity/v3/projects/12ae024a6c36462e8ede739817cee650 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:50:50.568251 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3bc9db-941c-4782-aa7e-e7df2bac4954 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:50.569720 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-be3bc9db-941c-4782-aa7e-e7df2bac4954 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:50:50.640852 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3bc9db-941c-4782-aa7e-e7df2bac4954 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:50.687553 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:50:50.702498 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f693b20-ac00-432b-b8e5-50f70f1dfbae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:50.703556 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f693b20-ac00-432b-b8e5-50f70f1dfbae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:50.703716 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f693b20-ac00-432b-b8e5-50f70f1dfbae admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:50.703990 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f693b20-ac00-432b-b8e5-50f70f1dfbae admin admin] PATH_INFO: `/v3/users/bb9169345c804ebe828604966f24c805` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:50.705274 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f693b20-ac00-432b-b8e5-50f70f1dfbae admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb9169345c804ebe828604966f24c805)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:50.707188 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f693b20-ac00-432b-b8e5-50f70f1dfbae admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:50.766276 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1921825d-9250-421d-83c4-bf0c2b2d1504 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:50.768283 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-6f693b20-ac00-432b-b8e5-50f70f1dfbae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb9169345c804ebe828604966f24c805'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:50.769086 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-6f693b20-ac00-432b-b8e5-50f70f1dfbae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb9169345c804ebe828604966f24c805'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:50.769398 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-1921825d-9250-421d-83c4-bf0c2b2d1504 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:50:50.780426 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-6f693b20-ac00-432b-b8e5-50f70f1dfbae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb9169345c804ebe828604966f24c805'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:50.781079 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-6f693b20-ac00-432b-b8e5-50f70f1dfbae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb9169345c804ebe828604966f24c805'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:50.787983 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1921825d-9250-421d-83c4-bf0c2b2d1504 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:50.793201 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b732317e-9b47-4fb1-b1e3-1b0555915c2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:50.794214 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b732317e-9b47-4fb1-b1e3-1b0555915c2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:50.794459 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b732317e-9b47-4fb1-b1e3-1b0555915c2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:50.794937 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b732317e-9b47-4fb1-b1e3-1b0555915c2f admin admin] PATH_INFO: `/v3/projects/9ade688548524aa6b7a7f01300416c5e` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:50.796122 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b732317e-9b47-4fb1-b1e3-1b0555915c2f admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ade688548524aa6b7a7f01300416c5e)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:50.796768 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-6f693b20-ac00-432b-b8e5-50f70f1dfbae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb9169345c804ebe828604966f24c805'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:50.798067 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b732317e-9b47-4fb1-b1e3-1b0555915c2f admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:50.805953 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=76,keystone:DELETE=21,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:50:50.812736 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1212/2443] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jun 16 17:50:50 2022] DELETE /identity/v3/users/bb9169345c804ebe828604966f24c805 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:50:50.851662 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-b732317e-9b47-4fb1-b1e3-1b0555915c2f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ade688548524aa6b7a7f01300416c5e', 'request_id': 'req-b732317e-9b47-4fb1-b1e3-1b0555915c2f'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:50.860631 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-b732317e-9b47-4fb1-b1e3-1b0555915c2f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ade688548524aa6b7a7f01300416c5e', 'request_id': 'req-b732317e-9b47-4fb1-b1e3-1b0555915c2f'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:50.872915 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-b732317e-9b47-4fb1-b1e3-1b0555915c2f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ade688548524aa6b7a7f01300416c5e', 'request_id': 'req-b732317e-9b47-4fb1-b1e3-1b0555915c2f'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:50.885548 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-b732317e-9b47-4fb1-b1e3-1b0555915c2f admin admin] The token cache is being invalidate because project 9ade688548524aa6b7a7f01300416c5e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:50:50.885548 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-b732317e-9b47-4fb1-b1e3-1b0555915c2f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:50.885968 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-b732317e-9b47-4fb1-b1e3-1b0555915c2f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:50.888174 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1232/2444] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jun 16 17:50:50 2022] DELETE /identity/v3/projects/9ade688548524aa6b7a7f01300416c5e => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:50:50.892827 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a370e700-d36b-411b-b3b4-f8ad9251ac3a None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:50.893125 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a370e700-d36b-411b-b3b4-f8ad9251ac3a None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:50.893401 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a370e700-d36b-411b-b3b4-f8ad9251ac3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:50.913714 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-a370e700-d36b-411b-b3b4-f8ad9251ac3a None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:50:50.915006 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-a370e700-d36b-411b-b3b4-f8ad9251ac3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:50:50.982820 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1213/2445] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:50:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:50:50.991569 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea3dbdf-2e59-4c2a-97ac-38a5170274a2 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:50.991862 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea3dbdf-2e59-4c2a-97ac-38a5170274a2 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:50.992141 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea3dbdf-2e59-4c2a-97ac-38a5170274a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:51.018302 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-9ea3dbdf-2e59-4c2a-97ac-38a5170274a2 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:50:51.019531 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-9ea3dbdf-2e59-4c2a-97ac-38a5170274a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:50:51.028139 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1233/2446] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:50:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:50:51.032409 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a370e700-d36b-411b-b3b4-f8ad9251ac3a None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:51.043169 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a370e700-d36b-411b-b3b4-f8ad9251ac3a None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:51.048408 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76147837-6a8c-430f-a999-50c22fd943ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:51.049143 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76147837-6a8c-430f-a999-50c22fd943ae admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:51.049483 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76147837-6a8c-430f-a999-50c22fd943ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:51.049791 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76147837-6a8c-430f-a999-50c22fd943ae admin admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:51.050555 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76147837-6a8c-430f-a999-50c22fd943ae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:50:51.050865 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76147837-6a8c-430f-a999-50c22fd943ae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:51.053177 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76147837-6a8c-430f-a999-50c22fd943ae admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:51.065161 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1214/2447] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jun 16 17:50:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:50:51.069567 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea3dbdf-2e59-4c2a-97ac-38a5170274a2 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:51.079512 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea3dbdf-2e59-4c2a-97ac-38a5170274a2 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:51.085058 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2549f173-ba2e-4b5d-a1c8-471d54fac307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:51.085722 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2549f173-ba2e-4b5d-a1c8-471d54fac307 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:51.086006 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2549f173-ba2e-4b5d-a1c8-471d54fac307 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:51.086284 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2549f173-ba2e-4b5d-a1c8-471d54fac307 admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:51.086979 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2549f173-ba2e-4b5d-a1c8-471d54fac307 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:51.088951 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2549f173-ba2e-4b5d-a1c8-471d54fac307 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:51.102042 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76147837-6a8c-430f-a999-50c22fd943ae admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:51.104460 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-76147837-6a8c-430f-a999-50c22fd943ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:50:51.118450 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76147837-6a8c-430f-a999-50c22fd943ae admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:51.124027 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1234/2448] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jun 16 17:50:51 2022] POST /identity/v3/projects => generated 381 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:50:51.124191 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3501096-8367-4996-aff6-053a50df0ded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:51.124191 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3501096-8367-4996-aff6-053a50df0ded admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:51.124437 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3501096-8367-4996-aff6-053a50df0ded admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:51.124722 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3501096-8367-4996-aff6-053a50df0ded admin admin] PATH_INFO: `/v3/projects/e4ba1f116cbf47abab03ecc1e6b6cc69` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:51.125284 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2549f173-ba2e-4b5d-a1c8-471d54fac307 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:51.125979 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3501096-8367-4996-aff6-053a50df0ded admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4ba1f116cbf47abab03ecc1e6b6cc69)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:51.128658 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3501096-8367-4996-aff6-053a50df0ded admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:51.190376 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-e3501096-8367-4996-aff6-053a50df0ded admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4ba1f116cbf47abab03ecc1e6b6cc69', 'request_id': 'req-e3501096-8367-4996-aff6-053a50df0ded'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:51.192833 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:50:51.201793 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-e3501096-8367-4996-aff6-053a50df0ded admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4ba1f116cbf47abab03ecc1e6b6cc69', 'request_id': 'req-e3501096-8367-4996-aff6-053a50df0ded'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:51.209724 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-e3501096-8367-4996-aff6-053a50df0ded admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4ba1f116cbf47abab03ecc1e6b6cc69', 'request_id': 'req-e3501096-8367-4996-aff6-053a50df0ded'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:51.221552 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2549f173-ba2e-4b5d-a1c8-471d54fac307 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:51.225188 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-e3501096-8367-4996-aff6-053a50df0ded admin admin] The token cache is being invalidate because project e4ba1f116cbf47abab03ecc1e6b6cc69 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80357) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:50:51.225587 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-e3501096-8367-4996-aff6-053a50df0ded admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:51.226576 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-e3501096-8367-4996-aff6-053a50df0ded admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:51.228479 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1215/2449] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jun 16 17:50:51 2022] DELETE /identity/v3/projects/e4ba1f116cbf47abab03ecc1e6b6cc69 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:50:51.234013 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dec73d2-775f-469c-a924-7b9ea46d3424 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:51.234461 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dec73d2-775f-469c-a924-7b9ea46d3424 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:51.234809 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dec73d2-775f-469c-a924-7b9ea46d3424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:51.254992 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-0dec73d2-775f-469c-a924-7b9ea46d3424 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:50:51.255957 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-0dec73d2-775f-469c-a924-7b9ea46d3424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:50:51.290673 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10d7ca9-6ef3-4d63-8373-80bcb24f37d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:51.291393 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10d7ca9-6ef3-4d63-8373-80bcb24f37d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:51.291681 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10d7ca9-6ef3-4d63-8373-80bcb24f37d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:51.291957 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10d7ca9-6ef3-4d63-8373-80bcb24f37d9 admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:51.292811 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10d7ca9-6ef3-4d63-8373-80bcb24f37d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:51.295983 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10d7ca9-6ef3-4d63-8373-80bcb24f37d9 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:51.326211 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1235/2450] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jun 16 17:50:51 2022] POST /identity/v3/users => generated 291 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:50:51.329049 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1216/2451] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:50:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:50:51.329671 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10d7ca9-6ef3-4d63-8373-80bcb24f37d9 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:51.331439 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-a10d7ca9-6ef3-4d63-8373-80bcb24f37d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:50:51.337945 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628020aa-07f1-438b-b8ba-06d14f9608fc None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:51.338517 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628020aa-07f1-438b-b8ba-06d14f9608fc None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:51.338517 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628020aa-07f1-438b-b8ba-06d14f9608fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:51.340217 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:50:51.346683 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10d7ca9-6ef3-4d63-8373-80bcb24f37d9 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:51.353755 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9b63bf-1b5b-4f3f-b145-7583eb097bd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:51.354561 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9b63bf-1b5b-4f3f-b145-7583eb097bd3 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:51.355454 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9b63bf-1b5b-4f3f-b145-7583eb097bd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:51.355454 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9b63bf-1b5b-4f3f-b145-7583eb097bd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:51.355974 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9b63bf-1b5b-4f3f-b145-7583eb097bd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:51.358567 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9b63bf-1b5b-4f3f-b145-7583eb097bd3 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:51.366281 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-628020aa-07f1-438b-b8ba-06d14f9608fc None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:50:51.367538 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-628020aa-07f1-438b-b8ba-06d14f9608fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:50:51.372452 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1236/2452] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:50:51 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 17:50:51.373500 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9b63bf-1b5b-4f3f-b145-7583eb097bd3 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:51.375947 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1217/2453] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:50:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:50:51.380948 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628020aa-07f1-438b-b8ba-06d14f9608fc None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:51.385298 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9b63bf-1b5b-4f3f-b145-7583eb097bd3 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:51.391114 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f02d976-72ad-414c-8862-aceb6bec280d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:51.391413 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628020aa-07f1-438b-b8ba-06d14f9608fc None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:51.391793 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f02d976-72ad-414c-8862-aceb6bec280d admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:51.392103 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f02d976-72ad-414c-8862-aceb6bec280d admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:51.392407 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f02d976-72ad-414c-8862-aceb6bec280d admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:51.393330 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f02d976-72ad-414c-8862-aceb6bec280d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:51.395637 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f02d976-72ad-414c-8862-aceb6bec280d admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:51.397774 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51104e11-a220-4174-b8a7-d14253de97f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:51.398462 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51104e11-a220-4174-b8a7-d14253de97f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:51.398749 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51104e11-a220-4174-b8a7-d14253de97f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:51.399027 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51104e11-a220-4174-b8a7-d14253de97f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:51.399760 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51104e11-a220-4174-b8a7-d14253de97f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:50:51.400064 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51104e11-a220-4174-b8a7-d14253de97f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:51.401875 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51104e11-a220-4174-b8a7-d14253de97f0 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:51.412006 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1237/2454] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:50:51 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:50:51.413938 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f02d976-72ad-414c-8862-aceb6bec280d admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:51.416406 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1218/2455] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jun 16 17:50:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:50:51.420189 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51104e11-a220-4174-b8a7-d14253de97f0 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:51.428514 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f02d976-72ad-414c-8862-aceb6bec280d admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:51.431188 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51104e11-a220-4174-b8a7-d14253de97f0 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:51.433514 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7232e231-89ef-46dd-b1ee-365e334144c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:51.434522 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7232e231-89ef-46dd-b1ee-365e334144c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:51.434811 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7232e231-89ef-46dd-b1ee-365e334144c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:51.435093 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7232e231-89ef-46dd-b1ee-365e334144c3 admin admin] PATH_INFO: `/v3/projects/a295b3cc1836436cbe5819058e140cac/users/480dbb7914f54383b99fc6b29e0bffc8/roles/0be2075946f54c8485f1c275f746478a` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:51.436023 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41c7e67-d592-4d7b-9065-9d439acb35c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:51.436783 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41c7e67-d592-4d7b-9065-9d439acb35c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:51.437072 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41c7e67-d592-4d7b-9065-9d439acb35c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:51.437344 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41c7e67-d592-4d7b-9065-9d439acb35c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:51.438029 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41c7e67-d592-4d7b-9065-9d439acb35c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:51.439950 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41c7e67-d592-4d7b-9065-9d439acb35c0 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:51.454399 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7232e231-89ef-46dd-b1ee-365e334144c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a295b3cc1836436cbe5819058e140cac, user_id=480dbb7914f54383b99fc6b29e0bffc8, role_id=0be2075946f54c8485f1c275f746478a)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:51.457190 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7232e231-89ef-46dd-b1ee-365e334144c3 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:51.474365 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1219/2456] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jun 16 17:50:51 2022] POST /identity/v3/projects => generated 371 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:50:51.476325 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1238/2457] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:50:51 2022] PUT /identity/v3/projects/a295b3cc1836436cbe5819058e140cac/users/480dbb7914f54383b99fc6b29e0bffc8/roles/0be2075946f54c8485f1c275f746478a => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:50:51.479528 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41c7e67-d592-4d7b-9065-9d439acb35c0 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:51.490001 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42238439-c80b-4159-bb8c-b8a2bbafba5e None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:51.490566 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42238439-c80b-4159-bb8c-b8a2bbafba5e None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:51.490917 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42238439-c80b-4159-bb8c-b8a2bbafba5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:51.526325 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-42238439-c80b-4159-bb8c-b8a2bbafba5e None None] MFA Rules not processed for user `480dbb7914f54383b99fc6b29e0bffc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:50:51.527575 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-42238439-c80b-4159-bb8c-b8a2bbafba5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:50:51.549466 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41c7e67-d592-4d7b-9065-9d439acb35c0 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:51.554637 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8361f819-1cd0-4fff-a356-ac7161934b35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:51.555343 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8361f819-1cd0-4fff-a356-ac7161934b35 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:51.555633 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8361f819-1cd0-4fff-a356-ac7161934b35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:51.555911 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8361f819-1cd0-4fff-a356-ac7161934b35 admin admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:51.556689 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8361f819-1cd0-4fff-a356-ac7161934b35 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:51.558673 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8361f819-1cd0-4fff-a356-ac7161934b35 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:51.579880 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1220/2458] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jun 16 17:50:51 2022] POST /identity/v3/users => generated 286 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:50:51.584697 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8361f819-1cd0-4fff-a356-ac7161934b35 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:51.594226 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8361f819-1cd0-4fff-a356-ac7161934b35 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:51.594493 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1239/2459] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:50:51 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:50:51.598872 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42238439-c80b-4159-bb8c-b8a2bbafba5e None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:51.603001 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789727fe-37ad-4577-8cda-c2ebd93fdfa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:51.603647 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789727fe-37ad-4577-8cda-c2ebd93fdfa7 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:51.604012 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789727fe-37ad-4577-8cda-c2ebd93fdfa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:51.604204 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789727fe-37ad-4577-8cda-c2ebd93fdfa7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:51.604960 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789727fe-37ad-4577-8cda-c2ebd93fdfa7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:51.606587 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789727fe-37ad-4577-8cda-c2ebd93fdfa7 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:51.609869 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42238439-c80b-4159-bb8c-b8a2bbafba5e None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:51.615223 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e20b8c6-f123-4edb-a483-ed73c72fb5b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:51.615903 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e20b8c6-f123-4edb-a483-ed73c72fb5b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:51.616193 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e20b8c6-f123-4edb-a483-ed73c72fb5b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:51.616515 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e20b8c6-f123-4edb-a483-ed73c72fb5b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:51.617561 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e20b8c6-f123-4edb-a483-ed73c72fb5b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:51.617690 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1221/2460] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:50:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 17:50:51.621130 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789727fe-37ad-4577-8cda-c2ebd93fdfa7 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:51.622143 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e20b8c6-f123-4edb-a483-ed73c72fb5b9 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:51.631169 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789727fe-37ad-4577-8cda-c2ebd93fdfa7 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:51.635792 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6808aa-3c5a-4693-bd65-92ff80ff3cb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:51.636723 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6808aa-3c5a-4693-bd65-92ff80ff3cb2 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:51.637007 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6808aa-3c5a-4693-bd65-92ff80ff3cb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:51.637418 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6808aa-3c5a-4693-bd65-92ff80ff3cb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:51.638135 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6808aa-3c5a-4693-bd65-92ff80ff3cb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:51.639974 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6808aa-3c5a-4693-bd65-92ff80ff3cb2 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:51.651828 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1222/2461] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:50:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 17:50:51.656605 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1240/2462] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jun 16 17:50:51 2022] POST /identity/v3/projects => generated 383 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:50:51.656748 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6808aa-3c5a-4693-bd65-92ff80ff3cb2 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:51.657758 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e20b8c6-f123-4edb-a483-ed73c72fb5b9 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:51.663024 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:INSERT=5,keystone:DELETE=3,keystone:SELECT=92 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:50:51.667244 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6808aa-3c5a-4693-bd65-92ff80ff3cb2 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:51.726069 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e20b8c6-f123-4edb-a483-ed73c72fb5b9 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:51.730978 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5e8308-7d4a-4893-b5ce-e68e00a86d2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:51.731645 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5e8308-7d4a-4893-b5ce-e68e00a86d2d admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:51.732063 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5e8308-7d4a-4893-b5ce-e68e00a86d2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:51.732412 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5e8308-7d4a-4893-b5ce-e68e00a86d2d admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:51.733196 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5e8308-7d4a-4893-b5ce-e68e00a86d2d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:51.735100 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5e8308-7d4a-4893-b5ce-e68e00a86d2d admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:51.737613 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8582953-9d95-46b0-aefa-115028a09708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:51.738509 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8582953-9d95-46b0-aefa-115028a09708 admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:51.738842 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8582953-9d95-46b0-aefa-115028a09708 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:51.739117 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8582953-9d95-46b0-aefa-115028a09708 admin admin] PATH_INFO: `/v3/projects/6029565b74cd44ad830c8972988b87b0/users/64ee41487b5c41c1935394d69db5454a/roles/0be2075946f54c8485f1c275f746478a` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:51.755782 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8582953-9d95-46b0-aefa-115028a09708 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6029565b74cd44ad830c8972988b87b0, user_id=64ee41487b5c41c1935394d69db5454a, role_id=0be2075946f54c8485f1c275f746478a)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:51.757395 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1241/2463] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jun 16 17:50:51 2022] POST /identity/v3/users => generated 298 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:50:51.759305 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8582953-9d95-46b0-aefa-115028a09708 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:51.761755 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5e8308-7d4a-4893-b5ce-e68e00a86d2d admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:51.773602 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5e8308-7d4a-4893-b5ce-e68e00a86d2d admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:51.774801 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1223/2464] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:50:51 2022] PUT /identity/v3/projects/6029565b74cd44ad830c8972988b87b0/users/64ee41487b5c41c1935394d69db5454a/roles/0be2075946f54c8485f1c275f746478a => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:50:51.779513 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059684a7-d1bc-44e8-b26d-e9dd94184521 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:51.779808 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059684a7-d1bc-44e8-b26d-e9dd94184521 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:51.780142 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059684a7-d1bc-44e8-b26d-e9dd94184521 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:51.784181 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:50:51.797669 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-059684a7-d1bc-44e8-b26d-e9dd94184521 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:50:51.798883 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-059684a7-d1bc-44e8-b26d-e9dd94184521 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:50:51.837153 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d831b550-90f6-4388-9529-7e764493b570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:51.837825 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d831b550-90f6-4388-9529-7e764493b570 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:51.838110 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d831b550-90f6-4388-9529-7e764493b570 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:51.838387 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d831b550-90f6-4388-9529-7e764493b570 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:51.839128 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d831b550-90f6-4388-9529-7e764493b570 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:51.840757 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d831b550-90f6-4388-9529-7e764493b570 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:51.851274 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1242/2465] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:50:51 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 17:50:51.855640 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d831b550-90f6-4388-9529-7e764493b570 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:51.860685 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1224/2466] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:50:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:50:51.866485 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d831b550-90f6-4388-9529-7e764493b570 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:51.871280 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c21a53-83cf-4f47-b699-d228a96a3bb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:51.872170 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c21a53-83cf-4f47-b699-d228a96a3bb5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:51.872487 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c21a53-83cf-4f47-b699-d228a96a3bb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:51.872767 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c21a53-83cf-4f47-b699-d228a96a3bb5 admin admin] PATH_INFO: `/v3/projects/a642cac73e464ef8bd2c0577c8a6349c/users/708d884e04444e1c92e6c0b6de99fd80/roles/3ca2dcbbd29d4a31a3ab78f2299bc239` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:51.876332 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059684a7-d1bc-44e8-b26d-e9dd94184521 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:51.877741 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-059684a7-d1bc-44e8-b26d-e9dd94184521 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:50:51.892972 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c21a53-83cf-4f47-b699-d228a96a3bb5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a642cac73e464ef8bd2c0577c8a6349c, user_id=708d884e04444e1c92e6c0b6de99fd80, role_id=3ca2dcbbd29d4a31a3ab78f2299bc239)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:51.896479 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c21a53-83cf-4f47-b699-d228a96a3bb5 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:51.912254 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1243/2467] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:50:51 2022] PUT /identity/v3/projects/a642cac73e464ef8bd2c0577c8a6349c/users/708d884e04444e1c92e6c0b6de99fd80/roles/3ca2dcbbd29d4a31a3ab78f2299bc239 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:50:51.916185 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c21a53-83cf-4f47-b699-d228a96a3bb5 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:51.938947 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059684a7-d1bc-44e8-b26d-e9dd94184521 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:51.979792 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c21a53-83cf-4f47-b699-d228a96a3bb5 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:51.980568 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:50:51.984688 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e3dc41-4ca9-40e8-b4df-1e9a39f56257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:51.985354 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e3dc41-4ca9-40e8-b4df-1e9a39f56257 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:51.985643 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e3dc41-4ca9-40e8-b4df-1e9a39f56257 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:51.985993 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e3dc41-4ca9-40e8-b4df-1e9a39f56257 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:51.986647 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e3dc41-4ca9-40e8-b4df-1e9a39f56257 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:51.988258 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e3dc41-4ca9-40e8-b4df-1e9a39f56257 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:51.991159 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc48a344-b1d8-48a6-8384-6e9810d37e21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:51.991804 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc48a344-b1d8-48a6-8384-6e9810d37e21 service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:51.992080 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc48a344-b1d8-48a6-8384-6e9810d37e21 service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:51.992349 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc48a344-b1d8-48a6-8384-6e9810d37e21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:51.999608 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1244/2468] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:50:51 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 17:50:52.002002 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc48a344-b1d8-48a6-8384-6e9810d37e21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:52.003474 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e3dc41-4ca9-40e8-b4df-1e9a39f56257 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:52.003631 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc48a344-b1d8-48a6-8384-6e9810d37e21 service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:52.014212 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e3dc41-4ca9-40e8-b4df-1e9a39f56257 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:52.014726 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1225/2469] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:50:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 17:50:52.019010 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd7dfdf-319b-49e6-8085-3a5a6d2a118a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:52.020551 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd7dfdf-319b-49e6-8085-3a5a6d2a118a admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:52.020890 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd7dfdf-319b-49e6-8085-3a5a6d2a118a admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:52.021406 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd7dfdf-319b-49e6-8085-3a5a6d2a118a admin admin] PATH_INFO: `/v3/projects/a642cac73e464ef8bd2c0577c8a6349c/users/708d884e04444e1c92e6c0b6de99fd80/roles/3ca2dcbbd29d4a31a3ab78f2299bc239` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:52.023467 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd7dfdf-319b-49e6-8085-3a5a6d2a118a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a642cac73e464ef8bd2c0577c8a6349c, user_id=708d884e04444e1c92e6c0b6de99fd80, role_id=3ca2dcbbd29d4a31a3ab78f2299bc239)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:52.026179 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd7dfdf-319b-49e6-8085-3a5a6d2a118a admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:52.038260 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1245/2470] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:50:52 2022] PUT /identity/v3/projects/a642cac73e464ef8bd2c0577c8a6349c/users/708d884e04444e1c92e6c0b6de99fd80/roles/3ca2dcbbd29d4a31a3ab78f2299bc239 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:50:52.049851 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad858416-ba21-4d39-bc1d-f6846c188a76 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:52.050136 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad858416-ba21-4d39-bc1d-f6846c188a76 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:52.050411 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad858416-ba21-4d39-bc1d-f6846c188a76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:52.086459 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-ad858416-ba21-4d39-bc1d-f6846c188a76 None None] MFA Rules not processed for user `708d884e04444e1c92e6c0b6de99fd80`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:50:52.087810 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-ad858416-ba21-4d39-bc1d-f6846c188a76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:50:52.166544 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1226/2471] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:50:52 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:50:52.179574 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd7dfdf-319b-49e6-8085-3a5a6d2a118a admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:52.181243 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-9fd7dfdf-319b-49e6-8085-3a5a6d2a118a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:50:52.250031 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd7dfdf-319b-49e6-8085-3a5a6d2a118a admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:52.255288 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a24cf37-3779-48b3-81da-b509c8d59a21 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24d5e2637b7043e08313909d5b06e329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:52.256251 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a24cf37-3779-48b3-81da-b509c8d59a21 service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:52.256722 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a24cf37-3779-48b3-81da-b509c8d59a21 service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:52.257137 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a24cf37-3779-48b3-81da-b509c8d59a21 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:52.267914 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a24cf37-3779-48b3-81da-b509c8d59a21 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:52.269393 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a24cf37-3779-48b3-81da-b509c8d59a21 service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:52.279446 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1246/2472] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:50:52 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:50:52.361676 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad858416-ba21-4d39-bc1d-f6846c188a76 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:52.373687 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad858416-ba21-4d39-bc1d-f6846c188a76 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:52.378346 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a822551f-e69a-42c0-8dc2-1ecb20a0ab87 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24d5e2637b7043e08313909d5b06e329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:52.378995 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a822551f-e69a-42c0-8dc2-1ecb20a0ab87 service cinder] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:52.379600 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a822551f-e69a-42c0-8dc2-1ecb20a0ab87 service cinder] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:52.380582 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a822551f-e69a-42c0-8dc2-1ecb20a0ab87 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:52.462674 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a822551f-e69a-42c0-8dc2-1ecb20a0ab87 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:52.464017 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a822551f-e69a-42c0-8dc2-1ecb20a0ab87 service cinder] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:52.474505 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1227/2473] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:50:52 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:50:52.698258 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f61f1d-20dc-428e-ac95-3d47424b3157 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:52.698590 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f61f1d-20dc-428e-ac95-3d47424b3157 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:52.698829 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f61f1d-20dc-428e-ac95-3d47424b3157 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:52.728566 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:50:52.744749 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-b2f61f1d-20dc-428e-ac95-3d47424b3157 None None] MFA Rules not processed for user `64ee41487b5c41c1935394d69db5454a`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:50:52.746516 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-b2f61f1d-20dc-428e-ac95-3d47424b3157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:50:52.746749 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a822551f-e69a-42c0-8dc2-1ecb20a0ab87 service cinder] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:52.818237 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a822551f-e69a-42c0-8dc2-1ecb20a0ab87 service cinder] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:52.823149 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3f49a6-145e-421f-8626-13b2f9fd9df6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:52.824125 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3f49a6-145e-421f-8626-13b2f9fd9df6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:52.824525 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3f49a6-145e-421f-8626-13b2f9fd9df6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:52.824872 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3f49a6-145e-421f-8626-13b2f9fd9df6 admin admin] PATH_INFO: `/v3/users/480dbb7914f54383b99fc6b29e0bffc8` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:52.826148 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3f49a6-145e-421f-8626-13b2f9fd9df6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=480dbb7914f54383b99fc6b29e0bffc8)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:52.828183 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3f49a6-145e-421f-8626-13b2f9fd9df6 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:52.828459 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1247/2474] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:50:52 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:50:52.833935 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f61f1d-20dc-428e-ac95-3d47424b3157 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:52.844435 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:50:52.846940 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f61f1d-20dc-428e-ac95-3d47424b3157 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:52.854100 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa09f29a-dbb2-458d-a60b-c8e5e68f54d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:52.854776 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa09f29a-dbb2-458d-a60b-c8e5e68f54d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:52.855286 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa09f29a-dbb2-458d-a60b-c8e5e68f54d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:52.855423 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa09f29a-dbb2-458d-a60b-c8e5e68f54d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:52.856025 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa09f29a-dbb2-458d-a60b-c8e5e68f54d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:52.857980 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa09f29a-dbb2-458d-a60b-c8e5e68f54d8 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:52.893858 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1248/2475] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jun 16 17:50:52 2022] POST /identity/v3/projects => generated 371 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:50:52.896452 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa09f29a-dbb2-458d-a60b-c8e5e68f54d8 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:52.900270 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-4e3f49a6-145e-421f-8626-13b2f9fd9df6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '480dbb7914f54383b99fc6b29e0bffc8'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:52.900978 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-4e3f49a6-145e-421f-8626-13b2f9fd9df6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '480dbb7914f54383b99fc6b29e0bffc8'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:52.908734 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-4e3f49a6-145e-421f-8626-13b2f9fd9df6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '480dbb7914f54383b99fc6b29e0bffc8'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:52.909345 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-4e3f49a6-145e-421f-8626-13b2f9fd9df6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '480dbb7914f54383b99fc6b29e0bffc8'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:52.922991 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-4e3f49a6-145e-421f-8626-13b2f9fd9df6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '480dbb7914f54383b99fc6b29e0bffc8'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:52.937260 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1228/2476] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jun 16 17:50:52 2022] DELETE /identity/v3/users/480dbb7914f54383b99fc6b29e0bffc8 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:50:52.943110 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b780401e-639a-433f-a40c-093e7df7c94e None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:52.943546 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b780401e-639a-433f-a40c-093e7df7c94e None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:52.943988 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b780401e-639a-433f-a40c-093e7df7c94e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:52.964848 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa09f29a-dbb2-458d-a60b-c8e5e68f54d8 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:52.965505 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-b780401e-639a-433f-a40c-093e7df7c94e None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:50:52.966681 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-b780401e-639a-433f-a40c-093e7df7c94e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:50:53.026282 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc62e229-faf4-42ca-bfdb-96fd7094c0ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:53.026933 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc62e229-faf4-42ca-bfdb-96fd7094c0ee admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:53.027219 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc62e229-faf4-42ca-bfdb-96fd7094c0ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:53.027501 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc62e229-faf4-42ca-bfdb-96fd7094c0ee admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:53.028226 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc62e229-faf4-42ca-bfdb-96fd7094c0ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:53.029712 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1229/2477] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:50:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:50:53.030197 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc62e229-faf4-42ca-bfdb-96fd7094c0ee admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:53.040341 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b780401e-639a-433f-a40c-093e7df7c94e None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:53.041766 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-b780401e-639a-433f-a40c-093e7df7c94e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:50:53.052945 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1249/2478] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jun 16 17:50:52 2022] POST /identity/v3/users => generated 292 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:50:53.057166 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc62e229-faf4-42ca-bfdb-96fd7094c0ee admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:53.059243 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-fc62e229-faf4-42ca-bfdb-96fd7094c0ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:50:53.072800 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc62e229-faf4-42ca-bfdb-96fd7094c0ee admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:53.079550 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c21932-9848-4542-aba8-4272783ca5d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:53.080344 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c21932-9848-4542-aba8-4272783ca5d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:53.080665 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c21932-9848-4542-aba8-4272783ca5d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:53.080947 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c21932-9848-4542-aba8-4272783ca5d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:53.081678 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c21932-9848-4542-aba8-4272783ca5d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:53.083498 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c21932-9848-4542-aba8-4272783ca5d7 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:53.093904 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1250/2479] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:50:53 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 17:50:53.098997 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c21932-9848-4542-aba8-4272783ca5d7 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:53.105458 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:50:53.109894 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c21932-9848-4542-aba8-4272783ca5d7 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:53.112514 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b780401e-639a-433f-a40c-093e7df7c94e None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:53.116945 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849dc6aa-b7c5-47f8-9a6f-9102ceb0f752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:53.117093 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6afa1c-c334-4757-b7cc-b6f1528b75a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:53.117607 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6afa1c-c334-4757-b7cc-b6f1528b75a2 service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:53.117888 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6afa1c-c334-4757-b7cc-b6f1528b75a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:53.118089 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849dc6aa-b7c5-47f8-9a6f-9102ceb0f752 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:53.118189 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6afa1c-c334-4757-b7cc-b6f1528b75a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:53.118494 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849dc6aa-b7c5-47f8-9a6f-9102ceb0f752 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:53.118838 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849dc6aa-b7c5-47f8-9a6f-9102ceb0f752 admin admin] PATH_INFO: `/v3/projects/b8c1855175d34071bd279ad580170e5f/users/e297987bfc8149b08429f27f8449fdd0/roles/3ca2dcbbd29d4a31a3ab78f2299bc239` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:53.127743 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6afa1c-c334-4757-b7cc-b6f1528b75a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:53.129267 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6afa1c-c334-4757-b7cc-b6f1528b75a2 service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:53.133755 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849dc6aa-b7c5-47f8-9a6f-9102ceb0f752 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8c1855175d34071bd279ad580170e5f, user_id=e297987bfc8149b08429f27f8449fdd0, role_id=3ca2dcbbd29d4a31a3ab78f2299bc239)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:53.136679 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849dc6aa-b7c5-47f8-9a6f-9102ceb0f752 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:53.139343 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1230/2480] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:50:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:50:53.151544 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1251/2481] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:50:53 2022] PUT /identity/v3/projects/b8c1855175d34071bd279ad580170e5f/users/e297987bfc8149b08429f27f8449fdd0/roles/3ca2dcbbd29d4a31a3ab78f2299bc239 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:50:53.155581 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6afa1c-c334-4757-b7cc-b6f1528b75a2 service neutron] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:53.166424 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6afa1c-c334-4757-b7cc-b6f1528b75a2 service neutron] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:53.199752 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:50:53.231415 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1402f358-db3a-4189-ae3e-b70307c677c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:53.232094 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1402f358-db3a-4189-ae3e-b70307c677c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:53.232393 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1402f358-db3a-4189-ae3e-b70307c677c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:53.232694 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1402f358-db3a-4189-ae3e-b70307c677c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:53.233421 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1402f358-db3a-4189-ae3e-b70307c677c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:53.235147 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1402f358-db3a-4189-ae3e-b70307c677c0 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:53.245828 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1231/2482] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:50:53 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:50:53.249992 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849dc6aa-b7c5-47f8-9a6f-9102ceb0f752 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:53.261515 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849dc6aa-b7c5-47f8-9a6f-9102ceb0f752 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:53.266286 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fae082-6223-4e0c-b6fb-9121d57da66e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:53.267189 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fae082-6223-4e0c-b6fb-9121d57da66e admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:53.267476 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fae082-6223-4e0c-b6fb-9121d57da66e admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:53.267865 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fae082-6223-4e0c-b6fb-9121d57da66e admin admin] PATH_INFO: `/v3/projects/b8c1855175d34071bd279ad580170e5f/users/e297987bfc8149b08429f27f8449fdd0/roles/3ca2dcbbd29d4a31a3ab78f2299bc239` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:53.270627 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fae082-6223-4e0c-b6fb-9121d57da66e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8c1855175d34071bd279ad580170e5f, user_id=e297987bfc8149b08429f27f8449fdd0, role_id=3ca2dcbbd29d4a31a3ab78f2299bc239)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:53.273450 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fae082-6223-4e0c-b6fb-9121d57da66e admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:53.284945 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1252/2483] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:50:53 2022] PUT /identity/v3/projects/b8c1855175d34071bd279ad580170e5f/users/e297987bfc8149b08429f27f8449fdd0/roles/3ca2dcbbd29d4a31a3ab78f2299bc239 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:50:53.440105 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1402f358-db3a-4189-ae3e-b70307c677c0 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:53.442544 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-1402f358-db3a-4189-ae3e-b70307c677c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:50:53.460823 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1402f358-db3a-4189-ae3e-b70307c677c0 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:53.529062 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b838d25-0781-4dbd-bdf9-45d78ffd6536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:53.529925 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b838d25-0781-4dbd-bdf9-45d78ffd6536 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:53.530210 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b838d25-0781-4dbd-bdf9-45d78ffd6536 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:53.530485 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b838d25-0781-4dbd-bdf9-45d78ffd6536 admin admin] PATH_INFO: `/v3/projects/a295b3cc1836436cbe5819058e140cac` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:53.531726 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b838d25-0781-4dbd-bdf9-45d78ffd6536 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a295b3cc1836436cbe5819058e140cac)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:53.533634 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b838d25-0781-4dbd-bdf9-45d78ffd6536 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:53.583079 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-6b838d25-0781-4dbd-bdf9-45d78ffd6536 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a295b3cc1836436cbe5819058e140cac', 'request_id': 'req-6b838d25-0781-4dbd-bdf9-45d78ffd6536'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:53.590338 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-6b838d25-0781-4dbd-bdf9-45d78ffd6536 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a295b3cc1836436cbe5819058e140cac', 'request_id': 'req-6b838d25-0781-4dbd-bdf9-45d78ffd6536'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:53.596104 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-6b838d25-0781-4dbd-bdf9-45d78ffd6536 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a295b3cc1836436cbe5819058e140cac', 'request_id': 'req-6b838d25-0781-4dbd-bdf9-45d78ffd6536'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:53.608279 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-6b838d25-0781-4dbd-bdf9-45d78ffd6536 admin admin] The token cache is being invalidate because project a295b3cc1836436cbe5819058e140cac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80357) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:50:53.608602 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-6b838d25-0781-4dbd-bdf9-45d78ffd6536 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:53.609370 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-6b838d25-0781-4dbd-bdf9-45d78ffd6536 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:53.610910 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1232/2484] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jun 16 17:50:53 2022] DELETE /identity/v3/projects/a295b3cc1836436cbe5819058e140cac => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:50:53.615829 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fae082-6223-4e0c-b6fb-9121d57da66e admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:53.618332 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-b7fae082-6223-4e0c-b6fb-9121d57da66e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:50:53.642933 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fae082-6223-4e0c-b6fb-9121d57da66e admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:53.727954 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d10beb0-edfe-43dc-97e1-054d4e464b70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:53.729439 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d10beb0-edfe-43dc-97e1-054d4e464b70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:53.729991 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d10beb0-edfe-43dc-97e1-054d4e464b70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:53.730481 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d10beb0-edfe-43dc-97e1-054d4e464b70 admin admin] PATH_INFO: `/v3/users/708d884e04444e1c92e6c0b6de99fd80` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:53.731978 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d10beb0-edfe-43dc-97e1-054d4e464b70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=708d884e04444e1c92e6c0b6de99fd80)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:53.734252 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d10beb0-edfe-43dc-97e1-054d4e464b70 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:53.794142 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-6d10beb0-edfe-43dc-97e1-054d4e464b70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '708d884e04444e1c92e6c0b6de99fd80'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:53.810394 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-6d10beb0-edfe-43dc-97e1-054d4e464b70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '708d884e04444e1c92e6c0b6de99fd80'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:53.811589 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-6d10beb0-edfe-43dc-97e1-054d4e464b70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '708d884e04444e1c92e6c0b6de99fd80'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:53.822379 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-6d10beb0-edfe-43dc-97e1-054d4e464b70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '708d884e04444e1c92e6c0b6de99fd80'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:53.833576 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-6d10beb0-edfe-43dc-97e1-054d4e464b70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '708d884e04444e1c92e6c0b6de99fd80'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:53.841896 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1253/2485] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jun 16 17:50:53 2022] DELETE /identity/v3/users/708d884e04444e1c92e6c0b6de99fd80 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:50:53.959344 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0224c9-592c-4651-893b-ffc5e64157fb None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:53.959641 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0224c9-592c-4651-893b-ffc5e64157fb None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:53.959985 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0224c9-592c-4651-893b-ffc5e64157fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:53.991603 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-be0224c9-592c-4651-893b-ffc5e64157fb None None] MFA Rules not processed for user `e297987bfc8149b08429f27f8449fdd0`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:50:53.992892 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-be0224c9-592c-4651-893b-ffc5e64157fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:50:54.061882 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:50:54.095388 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1233/2486] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:50:53 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:50:54.108617 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d10beb0-edfe-43dc-97e1-054d4e464b70 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:54.110052 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-6d10beb0-edfe-43dc-97e1-054d4e464b70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:50:54.183852 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:50:54.190127 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d10beb0-edfe-43dc-97e1-054d4e464b70 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:54.195124 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd11b979-cabe-4f9b-8c7b-1fd39eb26ee2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24d5e2637b7043e08313909d5b06e329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:54.195782 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd11b979-cabe-4f9b-8c7b-1fd39eb26ee2 service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:54.196065 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd11b979-cabe-4f9b-8c7b-1fd39eb26ee2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:54.196340 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd11b979-cabe-4f9b-8c7b-1fd39eb26ee2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:54.198698 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-cd11b979-cabe-4f9b-8c7b-1fd39eb26ee2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:50:54.211509 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0224c9-592c-4651-893b-ffc5e64157fb None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:54.213075 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-be0224c9-592c-4651-893b-ffc5e64157fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:50:54.286497 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd11b979-cabe-4f9b-8c7b-1fd39eb26ee2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:54.287939 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd11b979-cabe-4f9b-8c7b-1fd39eb26ee2 service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:54.293222 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0224c9-592c-4651-893b-ffc5e64157fb None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:54.298535 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67d61be-6476-4941-b805-e4faf4a1fb68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:54.299344 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67d61be-6476-4941-b805-e4faf4a1fb68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:54.299619 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67d61be-6476-4941-b805-e4faf4a1fb68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:54.299888 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67d61be-6476-4941-b805-e4faf4a1fb68 admin admin] PATH_INFO: `/v3/projects/a642cac73e464ef8bd2c0577c8a6349c` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:54.301632 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67d61be-6476-4941-b805-e4faf4a1fb68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a642cac73e464ef8bd2c0577c8a6349c)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:54.303583 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67d61be-6476-4941-b805-e4faf4a1fb68 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:54.304553 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1254/2487] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:50:54 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:50:54.361858 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-c67d61be-6476-4941-b805-e4faf4a1fb68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a642cac73e464ef8bd2c0577c8a6349c', 'request_id': 'req-c67d61be-6476-4941-b805-e4faf4a1fb68'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:54.373269 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-c67d61be-6476-4941-b805-e4faf4a1fb68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a642cac73e464ef8bd2c0577c8a6349c', 'request_id': 'req-c67d61be-6476-4941-b805-e4faf4a1fb68'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:54.382873 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-c67d61be-6476-4941-b805-e4faf4a1fb68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a642cac73e464ef8bd2c0577c8a6349c', 'request_id': 'req-c67d61be-6476-4941-b805-e4faf4a1fb68'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:54.396051 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-c67d61be-6476-4941-b805-e4faf4a1fb68 admin admin] The token cache is being invalidate because project a642cac73e464ef8bd2c0577c8a6349c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80357) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:50:54.396390 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-c67d61be-6476-4941-b805-e4faf4a1fb68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:54.397081 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-c67d61be-6476-4941-b805-e4faf4a1fb68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:50:54.398809 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1234/2488] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jun 16 17:50:54 2022] DELETE /identity/v3/projects/a642cac73e464ef8bd2c0577c8a6349c => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:50:54.404359 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf319fdb-c1a5-4411-bc41-5c7ef1190bfa None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:54.405136 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf319fdb-c1a5-4411-bc41-5c7ef1190bfa None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:54.405322 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf319fdb-c1a5-4411-bc41-5c7ef1190bfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:54.425942 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-cf319fdb-c1a5-4411-bc41-5c7ef1190bfa None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:50:54.427563 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-cf319fdb-c1a5-4411-bc41-5c7ef1190bfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:50:54.498625 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1255/2489] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:50:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:50:54.508526 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830e6997-234b-40bf-9f66-d78ec82087cf None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:54.509014 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830e6997-234b-40bf-9f66-d78ec82087cf None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:54.509326 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830e6997-234b-40bf-9f66-d78ec82087cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:54.531050 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-830e6997-234b-40bf-9f66-d78ec82087cf None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:50:54.532531 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-830e6997-234b-40bf-9f66-d78ec82087cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:50:54.542105 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1235/2490] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:50:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:50:54.546635 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf319fdb-c1a5-4411-bc41-5c7ef1190bfa None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:54.557343 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf319fdb-c1a5-4411-bc41-5c7ef1190bfa None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:54.562534 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09351dc4-c9dc-4b3e-bd7e-9ebcfbb7294c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:54.563311 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09351dc4-c9dc-4b3e-bd7e-9ebcfbb7294c admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:54.563675 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09351dc4-c9dc-4b3e-bd7e-9ebcfbb7294c admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:54.564028 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09351dc4-c9dc-4b3e-bd7e-9ebcfbb7294c admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:54.564875 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09351dc4-c9dc-4b3e-bd7e-9ebcfbb7294c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:50:54.565285 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09351dc4-c9dc-4b3e-bd7e-9ebcfbb7294c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:54.568047 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09351dc4-c9dc-4b3e-bd7e-9ebcfbb7294c admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:54.580294 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1256/2491] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jun 16 17:50:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:50:54.589803 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830e6997-234b-40bf-9f66-d78ec82087cf None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:54.605591 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830e6997-234b-40bf-9f66-d78ec82087cf None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:54.617703 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05a799f-f396-42ea-90f5-969334b2b80d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:54.618446 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05a799f-f396-42ea-90f5-969334b2b80d admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:54.618733 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05a799f-f396-42ea-90f5-969334b2b80d admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:54.619135 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05a799f-f396-42ea-90f5-969334b2b80d admin admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:54.620776 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05a799f-f396-42ea-90f5-969334b2b80d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:54.623787 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05a799f-f396-42ea-90f5-969334b2b80d admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:54.659309 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1236/2492] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jun 16 17:50:54 2022] POST /identity/v3/projects => generated 385 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:50:54.665306 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09351dc4-c9dc-4b3e-bd7e-9ebcfbb7294c admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:54.679447 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09351dc4-c9dc-4b3e-bd7e-9ebcfbb7294c admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:54.742461 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085b39be-749f-4b85-b19c-e87c8839a77e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:54.743139 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085b39be-749f-4b85-b19c-e87c8839a77e admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:54.743423 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085b39be-749f-4b85-b19c-e87c8839a77e admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:54.743707 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085b39be-749f-4b85-b19c-e87c8839a77e admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:54.744413 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085b39be-749f-4b85-b19c-e87c8839a77e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:54.746349 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085b39be-749f-4b85-b19c-e87c8839a77e admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:54.767882 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1257/2493] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jun 16 17:50:54 2022] POST /identity/v3/users => generated 293 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:50:54.773031 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05a799f-f396-42ea-90f5-969334b2b80d admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:54.784330 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05a799f-f396-42ea-90f5-969334b2b80d admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:54.790612 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdf879e-ef4d-441f-b17f-9f9bae976d48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:54.791387 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdf879e-ef4d-441f-b17f-9f9bae976d48 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:54.791787 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdf879e-ef4d-441f-b17f-9f9bae976d48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:54.792130 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdf879e-ef4d-441f-b17f-9f9bae976d48 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:54.792968 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdf879e-ef4d-441f-b17f-9f9bae976d48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:54.794660 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdf879e-ef4d-441f-b17f-9f9bae976d48 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:54.806616 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1237/2494] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:50:54 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 17:50:54.810572 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085b39be-749f-4b85-b19c-e87c8839a77e admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:54.821327 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085b39be-749f-4b85-b19c-e87c8839a77e admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:54.827932 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082a5ad2-14b9-44fa-876d-71c620e70396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:54.828756 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082a5ad2-14b9-44fa-876d-71c620e70396 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:54.829042 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082a5ad2-14b9-44fa-876d-71c620e70396 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:54.829323 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082a5ad2-14b9-44fa-876d-71c620e70396 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:54.830082 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082a5ad2-14b9-44fa-876d-71c620e70396 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:54.831683 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082a5ad2-14b9-44fa-876d-71c620e70396 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:54.842555 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1258/2495] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:50:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 17:50:54.846659 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdf879e-ef4d-441f-b17f-9f9bae976d48 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:54.857839 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdf879e-ef4d-441f-b17f-9f9bae976d48 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:54.862566 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1218228b-8b46-46fa-8ada-001c3e769a28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:54.863462 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1218228b-8b46-46fa-8ada-001c3e769a28 admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:54.864782 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1218228b-8b46-46fa-8ada-001c3e769a28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:54.865081 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1218228b-8b46-46fa-8ada-001c3e769a28 admin admin] PATH_INFO: `/v3/projects/f48de7dd5c064e0e9fe08562b7b556d9/users/8912e4ab51b84c069f60f049a39668e7/roles/0be2075946f54c8485f1c275f746478a` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:54.880754 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1218228b-8b46-46fa-8ada-001c3e769a28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f48de7dd5c064e0e9fe08562b7b556d9, user_id=8912e4ab51b84c069f60f049a39668e7, role_id=0be2075946f54c8485f1c275f746478a)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:54.883414 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1218228b-8b46-46fa-8ada-001c3e769a28 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:54.897311 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1238/2496] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:50:54 2022] PUT /identity/v3/projects/f48de7dd5c064e0e9fe08562b7b556d9/users/8912e4ab51b84c069f60f049a39668e7/roles/0be2075946f54c8485f1c275f746478a => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:50:54.908852 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e2410f-d834-4950-bfe7-756b878a8a24 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:54.909144 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e2410f-d834-4950-bfe7-756b878a8a24 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:54.909424 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e2410f-d834-4950-bfe7-756b878a8a24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:54.921999 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:50:54.937145 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-a4e2410f-d834-4950-bfe7-756b878a8a24 None None] MFA Rules not processed for user `8912e4ab51b84c069f60f049a39668e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:50:54.938332 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-a4e2410f-d834-4950-bfe7-756b878a8a24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:50:55.002476 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1259/2497] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:50:54 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:50:55.007532 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1218228b-8b46-46fa-8ada-001c3e769a28 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:55.022394 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:50:55.078575 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1218228b-8b46-46fa-8ada-001c3e769a28 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:55.083651 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515a6460-c6b2-4d7c-b5dc-042c622d2c1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:55.084309 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515a6460-c6b2-4d7c-b5dc-042c622d2c1c admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:55.084648 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515a6460-c6b2-4d7c-b5dc-042c622d2c1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:55.084954 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515a6460-c6b2-4d7c-b5dc-042c622d2c1c admin admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:55.085636 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515a6460-c6b2-4d7c-b5dc-042c622d2c1c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:55.087545 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515a6460-c6b2-4d7c-b5dc-042c622d2c1c admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:55.115033 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1239/2498] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jun 16 17:50:55 2022] POST /identity/v3/projects => generated 385 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:50:55.119306 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e2410f-d834-4950-bfe7-756b878a8a24 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:55.131822 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e2410f-d834-4950-bfe7-756b878a8a24 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:55.188479 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec63ac96-086b-4a93-b90d-c69e1c8f0896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:55.189153 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec63ac96-086b-4a93-b90d-c69e1c8f0896 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:55.189438 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec63ac96-086b-4a93-b90d-c69e1c8f0896 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:55.189720 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec63ac96-086b-4a93-b90d-c69e1c8f0896 admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:55.190416 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec63ac96-086b-4a93-b90d-c69e1c8f0896 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:55.193066 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec63ac96-086b-4a93-b90d-c69e1c8f0896 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:55.212567 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1260/2499] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jun 16 17:50:55 2022] POST /identity/v3/users => generated 299 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:50:55.216508 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515a6460-c6b2-4d7c-b5dc-042c622d2c1c admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:55.227278 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515a6460-c6b2-4d7c-b5dc-042c622d2c1c admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:55.231889 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96efca20-bd7f-4006-9f28-bed2b1a0145e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:55.232580 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96efca20-bd7f-4006-9f28-bed2b1a0145e admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:55.232872 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96efca20-bd7f-4006-9f28-bed2b1a0145e admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:55.233151 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96efca20-bd7f-4006-9f28-bed2b1a0145e admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:55.233882 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96efca20-bd7f-4006-9f28-bed2b1a0145e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:55.235482 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96efca20-bd7f-4006-9f28-bed2b1a0145e admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:55.246578 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1240/2500] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:50:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:50:55.250574 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec63ac96-086b-4a93-b90d-c69e1c8f0896 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:55.260586 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec63ac96-086b-4a93-b90d-c69e1c8f0896 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:55.265426 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141315f2-0412-4946-8af3-d8f3c4e6ab13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:55.266293 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141315f2-0412-4946-8af3-d8f3c4e6ab13 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:55.266573 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141315f2-0412-4946-8af3-d8f3c4e6ab13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:55.266846 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141315f2-0412-4946-8af3-d8f3c4e6ab13 admin admin] PATH_INFO: `/v3/projects/f3eb3275467047c3afb9fc3e8b98bd0a/users/9da62c612d12443c8e070cc422de5fbb/roles/3ca2dcbbd29d4a31a3ab78f2299bc239` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:55.281492 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141315f2-0412-4946-8af3-d8f3c4e6ab13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3eb3275467047c3afb9fc3e8b98bd0a, user_id=9da62c612d12443c8e070cc422de5fbb, role_id=3ca2dcbbd29d4a31a3ab78f2299bc239)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:55.284127 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141315f2-0412-4946-8af3-d8f3c4e6ab13 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:55.297283 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1261/2501] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:50:55 2022] PUT /identity/v3/projects/f3eb3275467047c3afb9fc3e8b98bd0a/users/9da62c612d12443c8e070cc422de5fbb/roles/3ca2dcbbd29d4a31a3ab78f2299bc239 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:50:55.301877 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96efca20-bd7f-4006-9f28-bed2b1a0145e admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:55.312270 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96efca20-bd7f-4006-9f28-bed2b1a0145e admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:55.377463 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4216241f-8dd5-4094-9fe2-c690945cf9e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:55.378047 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4216241f-8dd5-4094-9fe2-c690945cf9e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:55.378336 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4216241f-8dd5-4094-9fe2-c690945cf9e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:55.378614 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4216241f-8dd5-4094-9fe2-c690945cf9e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:55.379397 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4216241f-8dd5-4094-9fe2-c690945cf9e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:55.381015 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4216241f-8dd5-4094-9fe2-c690945cf9e0 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:55.392649 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1241/2502] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:50:55 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:50:55.397453 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141315f2-0412-4946-8af3-d8f3c4e6ab13 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:55.407982 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141315f2-0412-4946-8af3-d8f3c4e6ab13 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:55.413500 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f910275-db9a-4cd8-9260-44ec2a57da0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:55.414383 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f910275-db9a-4cd8-9260-44ec2a57da0e admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:55.414669 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f910275-db9a-4cd8-9260-44ec2a57da0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:55.414948 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f910275-db9a-4cd8-9260-44ec2a57da0e admin admin] PATH_INFO: `/v3/projects/f3eb3275467047c3afb9fc3e8b98bd0a/users/9da62c612d12443c8e070cc422de5fbb/roles/3ca2dcbbd29d4a31a3ab78f2299bc239` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:55.417238 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f910275-db9a-4cd8-9260-44ec2a57da0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3eb3275467047c3afb9fc3e8b98bd0a, user_id=9da62c612d12443c8e070cc422de5fbb, role_id=3ca2dcbbd29d4a31a3ab78f2299bc239)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:55.419719 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f910275-db9a-4cd8-9260-44ec2a57da0e admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:55.431044 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1262/2503] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:50:55 2022] PUT /identity/v3/projects/f3eb3275467047c3afb9fc3e8b98bd0a/users/9da62c612d12443c8e070cc422de5fbb/roles/3ca2dcbbd29d4a31a3ab78f2299bc239 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:50:55.441284 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d038b8-515b-41cf-af5a-28f380b9637d None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:55.441575 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d038b8-515b-41cf-af5a-28f380b9637d None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:55.442396 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d038b8-515b-41cf-af5a-28f380b9637d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:55.469101 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-57d038b8-515b-41cf-af5a-28f380b9637d None None] MFA Rules not processed for user `9da62c612d12443c8e070cc422de5fbb`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:50:55.470313 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-57d038b8-515b-41cf-af5a-28f380b9637d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:50:55.543174 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1242/2504] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:50:55 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:50:55.554115 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f910275-db9a-4cd8-9260-44ec2a57da0e admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:55.555818 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-1f910275-db9a-4cd8-9260-44ec2a57da0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:50:55.609552 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:50:55.614105 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f910275-db9a-4cd8-9260-44ec2a57da0e admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:55.619384 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c7ac2e-8540-4873-95f1-63b5d07ab0ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24d5e2637b7043e08313909d5b06e329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:55.620038 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c7ac2e-8540-4873-95f1-63b5d07ab0ca service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:55.620479 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c7ac2e-8540-4873-95f1-63b5d07ab0ca service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:55.620763 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c7ac2e-8540-4873-95f1-63b5d07ab0ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:55.679270 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c7ac2e-8540-4873-95f1-63b5d07ab0ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:55.680749 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c7ac2e-8540-4873-95f1-63b5d07ab0ca service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:55.689876 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1263/2505] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:50:55 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:50:55.707964 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d038b8-515b-41cf-af5a-28f380b9637d None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:55.709440 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-57d038b8-515b-41cf-af5a-28f380b9637d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:50:55.721648 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d038b8-515b-41cf-af5a-28f380b9637d None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:55.726205 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaa2e65-786a-40f0-b936-f37d57bb7b6d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24d5e2637b7043e08313909d5b06e329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:55.726878 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaa2e65-786a-40f0-b936-f37d57bb7b6d service cinder] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:55.727165 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaa2e65-786a-40f0-b936-f37d57bb7b6d service cinder] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:55.727446 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaa2e65-786a-40f0-b936-f37d57bb7b6d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:55.729257 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-efaa2e65-786a-40f0-b936-f37d57bb7b6d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:50:55.734519 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:50:55.810674 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efaa2e65-786a-40f0-b936-f37d57bb7b6d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:55.812119 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efaa2e65-786a-40f0-b936-f37d57bb7b6d service cinder] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:55.824632 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1243/2506] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:50:55 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:51:00.037429 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c7ac2e-8540-4873-95f1-63b5d07ab0ca service cinder] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:00.046366 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c7ac2e-8540-4873-95f1-63b5d07ab0ca service cinder] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:00.050034 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f605379e-1052-4494-81b9-14dab4834c77 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24d5e2637b7043e08313909d5b06e329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:00.050664 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f605379e-1052-4494-81b9-14dab4834c77 service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:00.051190 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f605379e-1052-4494-81b9-14dab4834c77 service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:00.051477 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f605379e-1052-4494-81b9-14dab4834c77 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:00.060629 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f605379e-1052-4494-81b9-14dab4834c77 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:00.062082 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f605379e-1052-4494-81b9-14dab4834c77 service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:00.071640 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1264/2507] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:51:00 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:51:04.992872 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaa2e65-786a-40f0-b936-f37d57bb7b6d service cinder] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:05.061532 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaa2e65-786a-40f0-b936-f37d57bb7b6d service cinder] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:05.066441 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9a06a4-e7a3-4844-b624-8213d0f7878b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:05.067643 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9a06a4-e7a3-4844-b624-8213d0f7878b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:05.067945 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9a06a4-e7a3-4844-b624-8213d0f7878b admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:05.068211 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9a06a4-e7a3-4844-b624-8213d0f7878b admin admin] PATH_INFO: `/v3/users/8912e4ab51b84c069f60f049a39668e7` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:05.069540 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9a06a4-e7a3-4844-b624-8213d0f7878b admin admin] RBAC: Authorizing `identity:delete_user(user_id=8912e4ab51b84c069f60f049a39668e7)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:05.071974 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9a06a4-e7a3-4844-b624-8213d0f7878b admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:05.152257 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-1f9a06a4-e7a3-4844-b624-8213d0f7878b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8912e4ab51b84c069f60f049a39668e7'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:05.152257 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-1f9a06a4-e7a3-4844-b624-8213d0f7878b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8912e4ab51b84c069f60f049a39668e7'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:05.157821 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-1f9a06a4-e7a3-4844-b624-8213d0f7878b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8912e4ab51b84c069f60f049a39668e7'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:05.158496 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-1f9a06a4-e7a3-4844-b624-8213d0f7878b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8912e4ab51b84c069f60f049a39668e7'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:05.171007 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-1f9a06a4-e7a3-4844-b624-8213d0f7878b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8912e4ab51b84c069f60f049a39668e7'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:05.185382 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1244/2508] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jun 16 17:51:04 2022] DELETE /identity/v3/users/8912e4ab51b84c069f60f049a39668e7 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:51:05.190002 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf9d37a-eab0-4958-860a-7bd08ba23297 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:05.196907 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf9d37a-eab0-4958-860a-7bd08ba23297 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:05.197400 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf9d37a-eab0-4958-860a-7bd08ba23297 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:05.217288 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-daf9d37a-eab0-4958-860a-7bd08ba23297 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:51:05.218515 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-daf9d37a-eab0-4958-860a-7bd08ba23297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:51:05.278524 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1265/2509] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:51:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:51:05.288938 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9a06a4-e7a3-4844-b624-8213d0f7878b admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:05.290318 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-1f9a06a4-e7a3-4844-b624-8213d0f7878b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:51:05.311571 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:51:05.349422 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9a06a4-e7a3-4844-b624-8213d0f7878b admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:05.353036 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4819b694-f32e-44aa-9729-46ed90fb59c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:05.353670 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4819b694-f32e-44aa-9729-46ed90fb59c6 service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:05.353949 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4819b694-f32e-44aa-9729-46ed90fb59c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:05.354221 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4819b694-f32e-44aa-9729-46ed90fb59c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:05.363576 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4819b694-f32e-44aa-9729-46ed90fb59c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:05.365027 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4819b694-f32e-44aa-9729-46ed90fb59c6 service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:05.374374 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1245/2510] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:51:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:51:05.660150 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf9d37a-eab0-4958-860a-7bd08ba23297 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:05.661609 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-daf9d37a-eab0-4958-860a-7bd08ba23297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:51:05.673960 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf9d37a-eab0-4958-860a-7bd08ba23297 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:05.679323 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ba48cc-2cd1-465e-8841-e267c01b4261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:05.680764 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ba48cc-2cd1-465e-8841-e267c01b4261 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:05.681389 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ba48cc-2cd1-465e-8841-e267c01b4261 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:05.681679 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ba48cc-2cd1-465e-8841-e267c01b4261 admin admin] PATH_INFO: `/v3/projects/f48de7dd5c064e0e9fe08562b7b556d9` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:05.682895 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ba48cc-2cd1-465e-8841-e267c01b4261 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f48de7dd5c064e0e9fe08562b7b556d9)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:05.692596 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ba48cc-2cd1-465e-8841-e267c01b4261 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:05.742135 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-22ba48cc-2cd1-465e-8841-e267c01b4261 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f48de7dd5c064e0e9fe08562b7b556d9', 'request_id': 'req-22ba48cc-2cd1-465e-8841-e267c01b4261'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:05.749528 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-22ba48cc-2cd1-465e-8841-e267c01b4261 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f48de7dd5c064e0e9fe08562b7b556d9', 'request_id': 'req-22ba48cc-2cd1-465e-8841-e267c01b4261'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:05.762421 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-22ba48cc-2cd1-465e-8841-e267c01b4261 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f48de7dd5c064e0e9fe08562b7b556d9', 'request_id': 'req-22ba48cc-2cd1-465e-8841-e267c01b4261'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:05.773272 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-22ba48cc-2cd1-465e-8841-e267c01b4261 admin admin] The token cache is being invalidate because project f48de7dd5c064e0e9fe08562b7b556d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:51:05.773564 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-22ba48cc-2cd1-465e-8841-e267c01b4261 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:05.774216 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-22ba48cc-2cd1-465e-8841-e267c01b4261 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:05.775784 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1266/2511] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jun 16 17:51:05 2022] DELETE /identity/v3/projects/f48de7dd5c064e0e9fe08562b7b556d9 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:51:05.779932 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4819b694-f32e-44aa-9729-46ed90fb59c6 service neutron] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:05.789826 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4819b694-f32e-44aa-9729-46ed90fb59c6 service neutron] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:05.851253 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306fff8f-a2e6-4a20-8af2-775dc72b0b53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:05.852137 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306fff8f-a2e6-4a20-8af2-775dc72b0b53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:05.852462 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306fff8f-a2e6-4a20-8af2-775dc72b0b53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:05.852742 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306fff8f-a2e6-4a20-8af2-775dc72b0b53 admin admin] PATH_INFO: `/v3/users/9da62c612d12443c8e070cc422de5fbb` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:05.854086 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306fff8f-a2e6-4a20-8af2-775dc72b0b53 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9da62c612d12443c8e070cc422de5fbb)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:05.855998 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306fff8f-a2e6-4a20-8af2-775dc72b0b53 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:05.907374 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-306fff8f-a2e6-4a20-8af2-775dc72b0b53 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9da62c612d12443c8e070cc422de5fbb'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:05.908029 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-306fff8f-a2e6-4a20-8af2-775dc72b0b53 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9da62c612d12443c8e070cc422de5fbb'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:05.915990 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-306fff8f-a2e6-4a20-8af2-775dc72b0b53 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9da62c612d12443c8e070cc422de5fbb'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:05.917024 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-306fff8f-a2e6-4a20-8af2-775dc72b0b53 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9da62c612d12443c8e070cc422de5fbb'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:05.930428 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-306fff8f-a2e6-4a20-8af2-775dc72b0b53 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9da62c612d12443c8e070cc422de5fbb'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:05.944177 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1246/2512] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jun 16 17:51:05 2022] DELETE /identity/v3/users/9da62c612d12443c8e070cc422de5fbb => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:51:06.244549 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ba48cc-2cd1-465e-8841-e267c01b4261 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:06.246255 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-22ba48cc-2cd1-465e-8841-e267c01b4261 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:51:06.271395 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ba48cc-2cd1-465e-8841-e267c01b4261 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:06.289210 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:51:06.367153 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54efca85-0686-494c-956a-7c9564e1f86c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:06.368137 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54efca85-0686-494c-956a-7c9564e1f86c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:06.368573 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54efca85-0686-494c-956a-7c9564e1f86c admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:06.368954 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54efca85-0686-494c-956a-7c9564e1f86c admin admin] PATH_INFO: `/v3/projects/f3eb3275467047c3afb9fc3e8b98bd0a` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:06.370297 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54efca85-0686-494c-956a-7c9564e1f86c admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3eb3275467047c3afb9fc3e8b98bd0a)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:06.372248 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54efca85-0686-494c-956a-7c9564e1f86c admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:06.424394 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-54efca85-0686-494c-956a-7c9564e1f86c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3eb3275467047c3afb9fc3e8b98bd0a', 'request_id': 'req-54efca85-0686-494c-956a-7c9564e1f86c'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:06.433898 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-54efca85-0686-494c-956a-7c9564e1f86c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3eb3275467047c3afb9fc3e8b98bd0a', 'request_id': 'req-54efca85-0686-494c-956a-7c9564e1f86c'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:06.440200 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-54efca85-0686-494c-956a-7c9564e1f86c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3eb3275467047c3afb9fc3e8b98bd0a', 'request_id': 'req-54efca85-0686-494c-956a-7c9564e1f86c'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:06.451000 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-54efca85-0686-494c-956a-7c9564e1f86c admin admin] The token cache is being invalidate because project f3eb3275467047c3afb9fc3e8b98bd0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:51:06.451293 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-54efca85-0686-494c-956a-7c9564e1f86c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:06.451950 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-54efca85-0686-494c-956a-7c9564e1f86c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:06.453737 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1267/2513] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jun 16 17:51:06 2022] DELETE /identity/v3/projects/f3eb3275467047c3afb9fc3e8b98bd0a => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:51:06.459778 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb14909-3074-4e2f-98b2-076c11e3a85e None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:06.460262 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb14909-3074-4e2f-98b2-076c11e3a85e None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:06.460353 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb14909-3074-4e2f-98b2-076c11e3a85e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:06.478994 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-ddb14909-3074-4e2f-98b2-076c11e3a85e None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:51:06.480201 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-ddb14909-3074-4e2f-98b2-076c11e3a85e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:51:06.521462 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:51:06.548679 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1247/2514] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:51:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:51:06.558419 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84d5597-facd-436f-af5c-fff9b1c33466 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:06.558706 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84d5597-facd-436f-af5c-fff9b1c33466 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:06.558986 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84d5597-facd-436f-af5c-fff9b1c33466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:06.577302 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-d84d5597-facd-436f-af5c-fff9b1c33466 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:51:06.578506 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-d84d5597-facd-436f-af5c-fff9b1c33466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:51:06.586773 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1268/2515] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:51:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:51:06.590892 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb14909-3074-4e2f-98b2-076c11e3a85e None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:06.602303 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb14909-3074-4e2f-98b2-076c11e3a85e None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:06.607172 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13337030-f069-4043-ad68-41d3de92b78b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:06.607781 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13337030-f069-4043-ad68-41d3de92b78b admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:06.608065 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13337030-f069-4043-ad68-41d3de92b78b admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:06.608343 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13337030-f069-4043-ad68-41d3de92b78b admin admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:06.609104 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13337030-f069-4043-ad68-41d3de92b78b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:51:06.609472 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13337030-f069-4043-ad68-41d3de92b78b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:06.611072 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13337030-f069-4043-ad68-41d3de92b78b admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:06.624070 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1248/2516] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jun 16 17:51:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:51:06.628777 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84d5597-facd-436f-af5c-fff9b1c33466 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:06.638317 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84d5597-facd-436f-af5c-fff9b1c33466 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:06.651753 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2378037-16c0-4820-b01b-f3967c9bb334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:06.652547 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2378037-16c0-4820-b01b-f3967c9bb334 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:06.652863 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2378037-16c0-4820-b01b-f3967c9bb334 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:06.653165 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2378037-16c0-4820-b01b-f3967c9bb334 admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:06.653903 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2378037-16c0-4820-b01b-f3967c9bb334 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:06.655813 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2378037-16c0-4820-b01b-f3967c9bb334 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:06.684508 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1269/2517] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jun 16 17:51:06 2022] POST /identity/v3/projects => generated 373 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:51:06.688655 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13337030-f069-4043-ad68-41d3de92b78b admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:06.698607 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13337030-f069-4043-ad68-41d3de92b78b admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:06.759388 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d422430c-4b36-4cd0-bece-cabdc3c271b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:06.760552 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d422430c-4b36-4cd0-bece-cabdc3c271b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:06.760552 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d422430c-4b36-4cd0-bece-cabdc3c271b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:06.760799 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d422430c-4b36-4cd0-bece-cabdc3c271b8 admin admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:06.761610 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d422430c-4b36-4cd0-bece-cabdc3c271b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:06.763574 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d422430c-4b36-4cd0-bece-cabdc3c271b8 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:06.792729 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1249/2518] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jun 16 17:51:06 2022] POST /identity/v3/users => generated 287 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:51:06.794468 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2378037-16c0-4820-b01b-f3967c9bb334 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:06.805776 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2378037-16c0-4820-b01b-f3967c9bb334 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:06.810485 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf5a2ae-31e3-4395-bc6e-acfbcb7ea0ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:06.811153 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf5a2ae-31e3-4395-bc6e-acfbcb7ea0ad admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:06.811440 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf5a2ae-31e3-4395-bc6e-acfbcb7ea0ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:06.811718 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf5a2ae-31e3-4395-bc6e-acfbcb7ea0ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:06.812474 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf5a2ae-31e3-4395-bc6e-acfbcb7ea0ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:06.814065 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf5a2ae-31e3-4395-bc6e-acfbcb7ea0ad admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:06.825133 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1270/2519] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:51:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:51:06.829425 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d422430c-4b36-4cd0-bece-cabdc3c271b8 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:06.842275 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d422430c-4b36-4cd0-bece-cabdc3c271b8 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:06.849868 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc22fb2-c8e1-474c-be17-47a63f31f118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:06.850622 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc22fb2-c8e1-474c-be17-47a63f31f118 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:06.850965 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc22fb2-c8e1-474c-be17-47a63f31f118 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:06.851303 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc22fb2-c8e1-474c-be17-47a63f31f118 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:06.852119 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc22fb2-c8e1-474c-be17-47a63f31f118 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:06.853864 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc22fb2-c8e1-474c-be17-47a63f31f118 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:06.865667 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1250/2520] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:51:06 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:51:06.872025 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf5a2ae-31e3-4395-bc6e-acfbcb7ea0ad admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:06.883098 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf5a2ae-31e3-4395-bc6e-acfbcb7ea0ad admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:06.888585 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc676cf2-43eb-4764-8fe5-4c395abbd1df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:06.889489 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc676cf2-43eb-4764-8fe5-4c395abbd1df admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:06.889770 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc676cf2-43eb-4764-8fe5-4c395abbd1df admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:06.890053 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc676cf2-43eb-4764-8fe5-4c395abbd1df admin admin] PATH_INFO: `/v3/projects/84bad7b567a041feb3e0571f159a96f8/users/73d7cf2facb04a608d8b7ecfe512b184/roles/0be2075946f54c8485f1c275f746478a` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:06.906848 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc676cf2-43eb-4764-8fe5-4c395abbd1df admin admin] RBAC: Authorizing `identity:create_grant(project_id=84bad7b567a041feb3e0571f159a96f8, user_id=73d7cf2facb04a608d8b7ecfe512b184, role_id=0be2075946f54c8485f1c275f746478a)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:06.909618 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc676cf2-43eb-4764-8fe5-4c395abbd1df admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:06.923751 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1271/2521] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:51:06 2022] PUT /identity/v3/projects/84bad7b567a041feb3e0571f159a96f8/users/73d7cf2facb04a608d8b7ecfe512b184/roles/0be2075946f54c8485f1c275f746478a => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:51:06.936257 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b8b677-1f44-4986-b58d-abfc4999b0ae None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:06.936586 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b8b677-1f44-4986-b58d-abfc4999b0ae None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:06.936871 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b8b677-1f44-4986-b58d-abfc4999b0ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:06.965065 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-81b8b677-1f44-4986-b58d-abfc4999b0ae None None] MFA Rules not processed for user `73d7cf2facb04a608d8b7ecfe512b184`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:51:06.966266 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-81b8b677-1f44-4986-b58d-abfc4999b0ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:51:07.029609 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1251/2522] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:51:06 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:51:07.041066 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc676cf2-43eb-4764-8fe5-4c395abbd1df admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:07.042897 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-dc676cf2-43eb-4764-8fe5-4c395abbd1df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:51:07.088027 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:51:07.103347 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc676cf2-43eb-4764-8fe5-4c395abbd1df admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:07.107327 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be875a2-c134-4ea7-8459-eeddca33a6a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24d5e2637b7043e08313909d5b06e329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:07.108013 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be875a2-c134-4ea7-8459-eeddca33a6a1 service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:07.108261 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be875a2-c134-4ea7-8459-eeddca33a6a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:07.108575 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be875a2-c134-4ea7-8459-eeddca33a6a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:07.118774 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be875a2-c134-4ea7-8459-eeddca33a6a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:07.120294 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be875a2-c134-4ea7-8459-eeddca33a6a1 service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:07.130321 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1272/2523] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:51:07 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:51:07.150708 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b8b677-1f44-4986-b58d-abfc4999b0ae None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:07.202162 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:51:07.232969 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b8b677-1f44-4986-b58d-abfc4999b0ae None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:07.237995 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cfd619-e3b1-4268-a0c5-c5b311926f2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:07.239628 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cfd619-e3b1-4268-a0c5-c5b311926f2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:07.239935 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cfd619-e3b1-4268-a0c5-c5b311926f2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:07.240210 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cfd619-e3b1-4268-a0c5-c5b311926f2c admin admin] PATH_INFO: `/v3/users/73d7cf2facb04a608d8b7ecfe512b184` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:07.241450 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cfd619-e3b1-4268-a0c5-c5b311926f2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=73d7cf2facb04a608d8b7ecfe512b184)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:07.243471 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cfd619-e3b1-4268-a0c5-c5b311926f2c admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:07.295745 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-41cfd619-e3b1-4268-a0c5-c5b311926f2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73d7cf2facb04a608d8b7ecfe512b184'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:07.296428 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-41cfd619-e3b1-4268-a0c5-c5b311926f2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73d7cf2facb04a608d8b7ecfe512b184'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:07.304360 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-41cfd619-e3b1-4268-a0c5-c5b311926f2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73d7cf2facb04a608d8b7ecfe512b184'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:07.305134 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-41cfd619-e3b1-4268-a0c5-c5b311926f2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73d7cf2facb04a608d8b7ecfe512b184'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:07.318082 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-41cfd619-e3b1-4268-a0c5-c5b311926f2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73d7cf2facb04a608d8b7ecfe512b184'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:07.331620 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1252/2524] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jun 16 17:51:07 2022] DELETE /identity/v3/users/73d7cf2facb04a608d8b7ecfe512b184 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:51:07.336118 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f408d6-59f6-456e-a68a-56e27d8b8eea None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:07.336605 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f408d6-59f6-456e-a68a-56e27d8b8eea None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:07.336765 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f408d6-59f6-456e-a68a-56e27d8b8eea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:07.354971 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-98f408d6-59f6-456e-a68a-56e27d8b8eea None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:51:07.356192 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-98f408d6-59f6-456e-a68a-56e27d8b8eea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:51:07.417221 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1273/2525] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:51:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:51:07.428396 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cfd619-e3b1-4268-a0c5-c5b311926f2c admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:07.429505 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-41cfd619-e3b1-4268-a0c5-c5b311926f2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:51:07.492581 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cfd619-e3b1-4268-a0c5-c5b311926f2c admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:07.496507 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0865653e-2c81-41ff-9ec7-069e429c63a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:07.497163 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0865653e-2c81-41ff-9ec7-069e429c63a1 service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:07.497448 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0865653e-2c81-41ff-9ec7-069e429c63a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:07.497729 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0865653e-2c81-41ff-9ec7-069e429c63a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:07.508211 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0865653e-2c81-41ff-9ec7-069e429c63a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:07.509912 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0865653e-2c81-41ff-9ec7-069e429c63a1 service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:07.522080 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1253/2526] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:51:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 17:51:07.859823 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f408d6-59f6-456e-a68a-56e27d8b8eea None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:07.862710 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-98f408d6-59f6-456e-a68a-56e27d8b8eea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:51:07.887857 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f408d6-59f6-456e-a68a-56e27d8b8eea None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:07.895808 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36692ba7-3b59-4008-b3f6-d9796b33e334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:07.895808 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36692ba7-3b59-4008-b3f6-d9796b33e334 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:07.895808 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36692ba7-3b59-4008-b3f6-d9796b33e334 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:07.895808 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36692ba7-3b59-4008-b3f6-d9796b33e334 admin admin] PATH_INFO: `/v3/projects/84bad7b567a041feb3e0571f159a96f8` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:07.897104 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36692ba7-3b59-4008-b3f6-d9796b33e334 admin admin] RBAC: Authorizing `identity:delete_project(project_id=84bad7b567a041feb3e0571f159a96f8)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:07.899012 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36692ba7-3b59-4008-b3f6-d9796b33e334 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:07.960617 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-36692ba7-3b59-4008-b3f6-d9796b33e334 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84bad7b567a041feb3e0571f159a96f8', 'request_id': 'req-36692ba7-3b59-4008-b3f6-d9796b33e334'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:07.968724 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-36692ba7-3b59-4008-b3f6-d9796b33e334 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84bad7b567a041feb3e0571f159a96f8', 'request_id': 'req-36692ba7-3b59-4008-b3f6-d9796b33e334'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:07.975556 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-36692ba7-3b59-4008-b3f6-d9796b33e334 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84bad7b567a041feb3e0571f159a96f8', 'request_id': 'req-36692ba7-3b59-4008-b3f6-d9796b33e334'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:07.987176 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-36692ba7-3b59-4008-b3f6-d9796b33e334 admin admin] The token cache is being invalidate because project 84bad7b567a041feb3e0571f159a96f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:51:07.987465 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-36692ba7-3b59-4008-b3f6-d9796b33e334 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:07.988184 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-36692ba7-3b59-4008-b3f6-d9796b33e334 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:07.990144 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1274/2527] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jun 16 17:51:07 2022] DELETE /identity/v3/projects/84bad7b567a041feb3e0571f159a96f8 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:51:07.994878 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfd8c74-c85d-4852-b92f-7a32ab0907b4 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:07.995172 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfd8c74-c85d-4852-b92f-7a32ab0907b4 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:07.995460 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfd8c74-c85d-4852-b92f-7a32ab0907b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:08.015025 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-0bfd8c74-c85d-4852-b92f-7a32ab0907b4 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:51:08.016238 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-0bfd8c74-c85d-4852-b92f-7a32ab0907b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:51:08.070910 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:51:08.081409 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1254/2528] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:51:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:51:08.089867 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f737075-099a-45cc-bacd-e624848aa2ec None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:08.090157 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f737075-099a-45cc-bacd-e624848aa2ec None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:08.090435 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f737075-099a-45cc-bacd-e624848aa2ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:08.110285 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-8f737075-099a-45cc-bacd-e624848aa2ec None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:51:08.111505 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-8f737075-099a-45cc-bacd-e624848aa2ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:51:08.120645 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1275/2529] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:51:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:51:08.124959 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfd8c74-c85d-4852-b92f-7a32ab0907b4 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:08.134916 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfd8c74-c85d-4852-b92f-7a32ab0907b4 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:08.139978 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25611d42-75bb-445e-a92a-8ea00bbeff41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:08.140710 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25611d42-75bb-445e-a92a-8ea00bbeff41 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:08.141178 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25611d42-75bb-445e-a92a-8ea00bbeff41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:08.141335 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25611d42-75bb-445e-a92a-8ea00bbeff41 admin admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:08.141973 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25611d42-75bb-445e-a92a-8ea00bbeff41 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:51:08.142390 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25611d42-75bb-445e-a92a-8ea00bbeff41 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:08.144021 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25611d42-75bb-445e-a92a-8ea00bbeff41 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:08.156774 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1255/2530] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jun 16 17:51:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:51:08.161776 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f737075-099a-45cc-bacd-e624848aa2ec None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:08.171100 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f737075-099a-45cc-bacd-e624848aa2ec None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:08.175794 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e18a57a-0569-4adf-858a-6d514971cd3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:08.177108 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e18a57a-0569-4adf-858a-6d514971cd3e admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:08.177470 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e18a57a-0569-4adf-858a-6d514971cd3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:08.177755 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e18a57a-0569-4adf-858a-6d514971cd3e admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:08.178448 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e18a57a-0569-4adf-858a-6d514971cd3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:08.180476 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e18a57a-0569-4adf-858a-6d514971cd3e admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:08.211255 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1276/2531] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jun 16 17:51:08 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:51:08.212611 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25611d42-75bb-445e-a92a-8ea00bbeff41 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:08.224165 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25611d42-75bb-445e-a92a-8ea00bbeff41 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:08.284033 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3600f9e9-c690-4da4-acbd-a46e4c5d15bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:08.284743 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3600f9e9-c690-4da4-acbd-a46e4c5d15bc admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:08.285033 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3600f9e9-c690-4da4-acbd-a46e4c5d15bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:08.285885 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3600f9e9-c690-4da4-acbd-a46e4c5d15bc admin admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:08.286606 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3600f9e9-c690-4da4-acbd-a46e4c5d15bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:08.288554 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3600f9e9-c690-4da4-acbd-a46e4c5d15bc admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:08.309617 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1256/2532] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jun 16 17:51:08 2022] POST /identity/v3/users => generated 292 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:51:08.313739 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e18a57a-0569-4adf-858a-6d514971cd3e admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:08.325747 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e18a57a-0569-4adf-858a-6d514971cd3e admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:08.330873 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bb1c83-3d35-41c3-9646-3e0b27d1107c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:08.331560 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bb1c83-3d35-41c3-9646-3e0b27d1107c admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:08.331847 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bb1c83-3d35-41c3-9646-3e0b27d1107c admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:08.332886 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bb1c83-3d35-41c3-9646-3e0b27d1107c admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:08.333725 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bb1c83-3d35-41c3-9646-3e0b27d1107c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:08.335399 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bb1c83-3d35-41c3-9646-3e0b27d1107c admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:08.347685 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1277/2533] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:51:08 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 17:51:08.352904 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3600f9e9-c690-4da4-acbd-a46e4c5d15bc admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:08.363849 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3600f9e9-c690-4da4-acbd-a46e4c5d15bc admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:08.369337 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c6fc90-bb9a-401c-bfe8-2f7ebd28f4b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:08.370019 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c6fc90-bb9a-401c-bfe8-2f7ebd28f4b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:08.370303 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c6fc90-bb9a-401c-bfe8-2f7ebd28f4b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:08.370581 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c6fc90-bb9a-401c-bfe8-2f7ebd28f4b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:08.371319 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c6fc90-bb9a-401c-bfe8-2f7ebd28f4b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:08.372992 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c6fc90-bb9a-401c-bfe8-2f7ebd28f4b8 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:08.391280 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1257/2534] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:51:08 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:51:08.392523 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bb1c83-3d35-41c3-9646-3e0b27d1107c admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:08.403410 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bb1c83-3d35-41c3-9646-3e0b27d1107c admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:08.408234 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63a84ab-a424-402a-8b5b-21e38771b9fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:08.409182 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63a84ab-a424-402a-8b5b-21e38771b9fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:08.409473 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63a84ab-a424-402a-8b5b-21e38771b9fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:08.409762 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63a84ab-a424-402a-8b5b-21e38771b9fe admin admin] PATH_INFO: `/v3/projects/da66c07c22de41d6ae3a5f5ddcb47af5/users/b6c8e9fd21c843d1aeaf80bafcf8899c/roles/0be2075946f54c8485f1c275f746478a` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:08.426894 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63a84ab-a424-402a-8b5b-21e38771b9fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=da66c07c22de41d6ae3a5f5ddcb47af5, user_id=b6c8e9fd21c843d1aeaf80bafcf8899c, role_id=0be2075946f54c8485f1c275f746478a)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:08.429859 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63a84ab-a424-402a-8b5b-21e38771b9fe admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:08.445157 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1278/2535] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:51:08 2022] PUT /identity/v3/projects/da66c07c22de41d6ae3a5f5ddcb47af5/users/b6c8e9fd21c843d1aeaf80bafcf8899c/roles/0be2075946f54c8485f1c275f746478a => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:51:08.458679 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f24d54c-79e4-4d94-808f-14655ac9a1bf None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:08.458971 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f24d54c-79e4-4d94-808f-14655ac9a1bf None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:08.459247 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f24d54c-79e4-4d94-808f-14655ac9a1bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:08.487901 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-5f24d54c-79e4-4d94-808f-14655ac9a1bf None None] MFA Rules not processed for user `b6c8e9fd21c843d1aeaf80bafcf8899c`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:51:08.490315 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-5f24d54c-79e4-4d94-808f-14655ac9a1bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:51:08.562464 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1258/2536] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:51:08 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:51:08.574874 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63a84ab-a424-402a-8b5b-21e38771b9fe admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:08.576643 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-b63a84ab-a424-402a-8b5b-21e38771b9fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:51:08.582146 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:51:08.637746 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63a84ab-a424-402a-8b5b-21e38771b9fe admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:08.641384 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8924d65a-2b18-4b1c-a7c1-44349311885d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24d5e2637b7043e08313909d5b06e329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:08.642024 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8924d65a-2b18-4b1c-a7c1-44349311885d service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:08.642304 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8924d65a-2b18-4b1c-a7c1-44349311885d service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:08.642576 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8924d65a-2b18-4b1c-a7c1-44349311885d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:08.651820 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8924d65a-2b18-4b1c-a7c1-44349311885d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:08.653306 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8924d65a-2b18-4b1c-a7c1-44349311885d service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:08.664208 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1279/2537] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:51:08 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:51:09.323368 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f24d54c-79e4-4d94-808f-14655ac9a1bf None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:09.325018 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-5f24d54c-79e4-4d94-808f-14655ac9a1bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:51:09.387004 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:51:09.393196 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f24d54c-79e4-4d94-808f-14655ac9a1bf None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:09.398540 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdd0cfc-2b8b-4dcf-9d2d-837580eba969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:09.399403 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdd0cfc-2b8b-4dcf-9d2d-837580eba969 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:09.399684 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdd0cfc-2b8b-4dcf-9d2d-837580eba969 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:09.399957 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdd0cfc-2b8b-4dcf-9d2d-837580eba969 admin admin] PATH_INFO: `/v3/users/64ee41487b5c41c1935394d69db5454a` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:09.401151 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdd0cfc-2b8b-4dcf-9d2d-837580eba969 admin admin] RBAC: Authorizing `identity:delete_user(user_id=64ee41487b5c41c1935394d69db5454a)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:09.403051 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdd0cfc-2b8b-4dcf-9d2d-837580eba969 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:09.447626 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-6bdd0cfc-2b8b-4dcf-9d2d-837580eba969 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64ee41487b5c41c1935394d69db5454a'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:09.448271 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-6bdd0cfc-2b8b-4dcf-9d2d-837580eba969 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64ee41487b5c41c1935394d69db5454a'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:09.455687 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-6bdd0cfc-2b8b-4dcf-9d2d-837580eba969 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64ee41487b5c41c1935394d69db5454a'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:09.456295 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-6bdd0cfc-2b8b-4dcf-9d2d-837580eba969 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64ee41487b5c41c1935394d69db5454a'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:09.475166 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-6bdd0cfc-2b8b-4dcf-9d2d-837580eba969 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64ee41487b5c41c1935394d69db5454a'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:09.489572 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1259/2538] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jun 16 17:51:09 2022] DELETE /identity/v3/users/64ee41487b5c41c1935394d69db5454a => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:51:09.661471 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8924d65a-2b18-4b1c-a7c1-44349311885d service cinder] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:09.673214 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8924d65a-2b18-4b1c-a7c1-44349311885d service cinder] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:09.748569 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6c8b3f-9679-4db1-8c02-6e7335e2f636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:09.749371 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6c8b3f-9679-4db1-8c02-6e7335e2f636 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:09.749637 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6c8b3f-9679-4db1-8c02-6e7335e2f636 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:09.749909 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6c8b3f-9679-4db1-8c02-6e7335e2f636 admin admin] PATH_INFO: `/v3/projects/6029565b74cd44ad830c8972988b87b0` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:09.751083 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6c8b3f-9679-4db1-8c02-6e7335e2f636 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6029565b74cd44ad830c8972988b87b0)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:09.753010 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6c8b3f-9679-4db1-8c02-6e7335e2f636 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:09.801511 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-eb6c8b3f-9679-4db1-8c02-6e7335e2f636 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6029565b74cd44ad830c8972988b87b0', 'request_id': 'req-eb6c8b3f-9679-4db1-8c02-6e7335e2f636'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:09.809343 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-eb6c8b3f-9679-4db1-8c02-6e7335e2f636 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6029565b74cd44ad830c8972988b87b0', 'request_id': 'req-eb6c8b3f-9679-4db1-8c02-6e7335e2f636'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:09.816108 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-eb6c8b3f-9679-4db1-8c02-6e7335e2f636 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6029565b74cd44ad830c8972988b87b0', 'request_id': 'req-eb6c8b3f-9679-4db1-8c02-6e7335e2f636'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:09.827669 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-eb6c8b3f-9679-4db1-8c02-6e7335e2f636 admin admin] The token cache is being invalidate because project 6029565b74cd44ad830c8972988b87b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:51:09.827989 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-eb6c8b3f-9679-4db1-8c02-6e7335e2f636 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:09.828691 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-eb6c8b3f-9679-4db1-8c02-6e7335e2f636 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:09.830304 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1280/2539] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jun 16 17:51:09 2022] DELETE /identity/v3/projects/6029565b74cd44ad830c8972988b87b0 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:51:09.835482 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdd0cfc-2b8b-4dcf-9d2d-837580eba969 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:09.837070 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-6bdd0cfc-2b8b-4dcf-9d2d-837580eba969 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:51:09.850975 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdd0cfc-2b8b-4dcf-9d2d-837580eba969 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:09.913246 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d8d431-7560-40a0-ab07-9d0793573a8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:09.914131 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d8d431-7560-40a0-ab07-9d0793573a8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:09.914420 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d8d431-7560-40a0-ab07-9d0793573a8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:09.914701 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d8d431-7560-40a0-ab07-9d0793573a8f admin admin] PATH_INFO: `/v3/users/e297987bfc8149b08429f27f8449fdd0` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:09.915926 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d8d431-7560-40a0-ab07-9d0793573a8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=e297987bfc8149b08429f27f8449fdd0)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:09.917935 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d8d431-7560-40a0-ab07-9d0793573a8f admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:09.970617 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-67d8d431-7560-40a0-ab07-9d0793573a8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e297987bfc8149b08429f27f8449fdd0'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:09.971297 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-67d8d431-7560-40a0-ab07-9d0793573a8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e297987bfc8149b08429f27f8449fdd0'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:09.982796 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-67d8d431-7560-40a0-ab07-9d0793573a8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e297987bfc8149b08429f27f8449fdd0'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:09.983538 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-67d8d431-7560-40a0-ab07-9d0793573a8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e297987bfc8149b08429f27f8449fdd0'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:09.996269 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-67d8d431-7560-40a0-ab07-9d0793573a8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e297987bfc8149b08429f27f8449fdd0'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:10.001002 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:51:10.013200 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1260/2540] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jun 16 17:51:09 2022] DELETE /identity/v3/users/e297987bfc8149b08429f27f8449fdd0 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:51:10.242675 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6c8b3f-9679-4db1-8c02-6e7335e2f636 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:10.244977 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-eb6c8b3f-9679-4db1-8c02-6e7335e2f636 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:51:10.270968 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6c8b3f-9679-4db1-8c02-6e7335e2f636 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:10.312077 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:51:10.342966 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e07e8b9-4619-4271-a950-48fceb7216ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:10.344013 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e07e8b9-4619-4271-a950-48fceb7216ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:10.344410 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e07e8b9-4619-4271-a950-48fceb7216ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:10.344828 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e07e8b9-4619-4271-a950-48fceb7216ff admin admin] PATH_INFO: `/v3/projects/b8c1855175d34071bd279ad580170e5f` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:10.346041 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e07e8b9-4619-4271-a950-48fceb7216ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8c1855175d34071bd279ad580170e5f)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:10.348161 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e07e8b9-4619-4271-a950-48fceb7216ff admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:10.407006 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-6e07e8b9-4619-4271-a950-48fceb7216ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8c1855175d34071bd279ad580170e5f', 'request_id': 'req-6e07e8b9-4619-4271-a950-48fceb7216ff'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:10.417986 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-6e07e8b9-4619-4271-a950-48fceb7216ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8c1855175d34071bd279ad580170e5f', 'request_id': 'req-6e07e8b9-4619-4271-a950-48fceb7216ff'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:10.425705 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-6e07e8b9-4619-4271-a950-48fceb7216ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8c1855175d34071bd279ad580170e5f', 'request_id': 'req-6e07e8b9-4619-4271-a950-48fceb7216ff'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:10.440566 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-6e07e8b9-4619-4271-a950-48fceb7216ff admin admin] The token cache is being invalidate because project b8c1855175d34071bd279ad580170e5f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:51:10.440889 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-6e07e8b9-4619-4271-a950-48fceb7216ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:10.442348 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-6e07e8b9-4619-4271-a950-48fceb7216ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:10.444266 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1281/2541] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jun 16 17:51:10 2022] DELETE /identity/v3/projects/b8c1855175d34071bd279ad580170e5f => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:51:10.449721 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7bf20e-2d3a-4ce3-a4f4-27bca5ec7910 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:10.450548 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7bf20e-2d3a-4ce3-a4f4-27bca5ec7910 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:10.450548 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7bf20e-2d3a-4ce3-a4f4-27bca5ec7910 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:10.479788 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-1e7bf20e-2d3a-4ce3-a4f4-27bca5ec7910 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:51:10.481985 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-1e7bf20e-2d3a-4ce3-a4f4-27bca5ec7910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:51:10.581818 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1261/2542] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:51:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:51:10.590744 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7de9abe-0331-443b-b66a-d3718d66a506 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:10.591186 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7de9abe-0331-443b-b66a-d3718d66a506 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:10.591400 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7de9abe-0331-443b-b66a-d3718d66a506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:10.618848 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-d7de9abe-0331-443b-b66a-d3718d66a506 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:51:10.620693 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-d7de9abe-0331-443b-b66a-d3718d66a506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:51:10.633487 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1282/2543] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:51:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:51:10.637916 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7bf20e-2d3a-4ce3-a4f4-27bca5ec7910 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:10.651204 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7bf20e-2d3a-4ce3-a4f4-27bca5ec7910 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:10.657348 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cde848-694d-4bb2-946d-08edba0809a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:10.658087 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cde848-694d-4bb2-946d-08edba0809a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:10.658380 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cde848-694d-4bb2-946d-08edba0809a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:10.658726 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cde848-694d-4bb2-946d-08edba0809a3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:10.659591 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cde848-694d-4bb2-946d-08edba0809a3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:51:10.659929 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cde848-694d-4bb2-946d-08edba0809a3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:10.661981 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cde848-694d-4bb2-946d-08edba0809a3 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:10.677814 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1262/2544] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jun 16 17:51:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:51:10.681908 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7de9abe-0331-443b-b66a-d3718d66a506 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:10.692295 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7de9abe-0331-443b-b66a-d3718d66a506 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:10.697495 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd89bbd-2653-425f-b587-d73bee29675f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:10.698152 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd89bbd-2653-425f-b587-d73bee29675f admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:10.698431 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd89bbd-2653-425f-b587-d73bee29675f admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:10.698703 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd89bbd-2653-425f-b587-d73bee29675f admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:10.699404 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd89bbd-2653-425f-b587-d73bee29675f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:10.701345 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd89bbd-2653-425f-b587-d73bee29675f admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:10.732130 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1283/2545] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jun 16 17:51:10 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:51:10.733335 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cde848-694d-4bb2-946d-08edba0809a3 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:10.744737 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cde848-694d-4bb2-946d-08edba0809a3 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:10.809042 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de18467-c7ac-49cd-9503-f7896811ced5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:10.809715 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de18467-c7ac-49cd-9503-f7896811ced5 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:10.809996 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de18467-c7ac-49cd-9503-f7896811ced5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:10.810270 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de18467-c7ac-49cd-9503-f7896811ced5 admin admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:10.810948 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de18467-c7ac-49cd-9503-f7896811ced5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:10.813048 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de18467-c7ac-49cd-9503-f7896811ced5 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:10.834694 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1263/2546] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jun 16 17:51:10 2022] POST /identity/v3/users => generated 287 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:51:10.838915 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd89bbd-2653-425f-b587-d73bee29675f admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:10.850306 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd89bbd-2653-425f-b587-d73bee29675f admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:10.855254 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dbb724-3991-45ca-a5dc-7757c91b8ead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:10.855919 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dbb724-3991-45ca-a5dc-7757c91b8ead admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:10.856202 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dbb724-3991-45ca-a5dc-7757c91b8ead admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:10.856521 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dbb724-3991-45ca-a5dc-7757c91b8ead admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:10.857244 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dbb724-3991-45ca-a5dc-7757c91b8ead admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:10.858835 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dbb724-3991-45ca-a5dc-7757c91b8ead admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:10.869929 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1284/2547] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:51:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:51:10.874027 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de18467-c7ac-49cd-9503-f7896811ced5 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:10.885153 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de18467-c7ac-49cd-9503-f7896811ced5 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:10.890300 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624cb95a-4cae-4895-bb67-0f0e726b87f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:10.890975 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624cb95a-4cae-4895-bb67-0f0e726b87f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:10.891266 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624cb95a-4cae-4895-bb67-0f0e726b87f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:10.891563 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624cb95a-4cae-4895-bb67-0f0e726b87f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:10.892301 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624cb95a-4cae-4895-bb67-0f0e726b87f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:10.893959 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624cb95a-4cae-4895-bb67-0f0e726b87f6 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:10.905422 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1264/2548] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:51:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:51:10.909621 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dbb724-3991-45ca-a5dc-7757c91b8ead admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:10.920164 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dbb724-3991-45ca-a5dc-7757c91b8ead admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:10.924960 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffff680-4481-4c13-97ab-403243c8c958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:10.926422 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffff680-4481-4c13-97ab-403243c8c958 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:10.926697 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffff680-4481-4c13-97ab-403243c8c958 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:10.927157 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffff680-4481-4c13-97ab-403243c8c958 admin admin] PATH_INFO: `/v3/projects/c6a903cccbd4459288d72e52410ffca1/users/c4bffefe8fb2475a96ea033d07c790f2/roles/0be2075946f54c8485f1c275f746478a` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:10.944711 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffff680-4481-4c13-97ab-403243c8c958 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6a903cccbd4459288d72e52410ffca1, user_id=c4bffefe8fb2475a96ea033d07c790f2, role_id=0be2075946f54c8485f1c275f746478a)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:10.947379 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffff680-4481-4c13-97ab-403243c8c958 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:10.962644 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1285/2549] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:51:10 2022] PUT /identity/v3/projects/c6a903cccbd4459288d72e52410ffca1/users/c4bffefe8fb2475a96ea033d07c790f2/roles/0be2075946f54c8485f1c275f746478a => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:51:10.968085 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd542e01-9357-44ac-bf50-c854d9e62e3c None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:10.968414 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd542e01-9357-44ac-bf50-c854d9e62e3c None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:10.968702 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd542e01-9357-44ac-bf50-c854d9e62e3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:11.003283 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-cd542e01-9357-44ac-bf50-c854d9e62e3c None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:51:11.005375 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-cd542e01-9357-44ac-bf50-c854d9e62e3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:51:11.067652 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:51:11.077605 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1265/2550] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:51:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:51:11.088274 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffff680-4481-4c13-97ab-403243c8c958 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:11.091056 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-cffff680-4481-4c13-97ab-403243c8c958 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:51:11.157083 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffff680-4481-4c13-97ab-403243c8c958 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:11.161493 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db34bbb-5c07-48ff-8b6c-a615c6ece348 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:11.162159 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db34bbb-5c07-48ff-8b6c-a615c6ece348 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:11.162439 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db34bbb-5c07-48ff-8b6c-a615c6ece348 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:11.162712 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db34bbb-5c07-48ff-8b6c-a615c6ece348 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:11.172429 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db34bbb-5c07-48ff-8b6c-a615c6ece348 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:11.173865 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db34bbb-5c07-48ff-8b6c-a615c6ece348 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:11.185256 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1286/2551] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:51:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 17:51:11.347058 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd542e01-9357-44ac-bf50-c854d9e62e3c None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:11.349696 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-cd542e01-9357-44ac-bf50-c854d9e62e3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:51:11.362971 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd542e01-9357-44ac-bf50-c854d9e62e3c None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:11.371829 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6909da-e857-446c-a64c-ff38ae62528c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:11.373209 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6909da-e857-446c-a64c-ff38ae62528c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:11.374733 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6909da-e857-446c-a64c-ff38ae62528c admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:11.375138 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6909da-e857-446c-a64c-ff38ae62528c admin admin] PATH_INFO: `/v3/users/b6c8e9fd21c843d1aeaf80bafcf8899c` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:11.377638 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6909da-e857-446c-a64c-ff38ae62528c admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6c8e9fd21c843d1aeaf80bafcf8899c)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:11.381534 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6909da-e857-446c-a64c-ff38ae62528c admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:11.493117 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-bb6909da-e857-446c-a64c-ff38ae62528c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6c8e9fd21c843d1aeaf80bafcf8899c'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:11.493699 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-bb6909da-e857-446c-a64c-ff38ae62528c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6c8e9fd21c843d1aeaf80bafcf8899c'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:11.503182 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-bb6909da-e857-446c-a64c-ff38ae62528c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6c8e9fd21c843d1aeaf80bafcf8899c'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:11.504067 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-bb6909da-e857-446c-a64c-ff38ae62528c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6c8e9fd21c843d1aeaf80bafcf8899c'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:11.549712 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-bb6909da-e857-446c-a64c-ff38ae62528c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6c8e9fd21c843d1aeaf80bafcf8899c'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:11.569545 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1266/2552] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jun 16 17:51:11 2022] DELETE /identity/v3/users/b6c8e9fd21c843d1aeaf80bafcf8899c => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:51:11.573380 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8981800d-b1b3-4d1d-9445-fd0728e02cc9 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:11.573907 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8981800d-b1b3-4d1d-9445-fd0728e02cc9 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:11.574425 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8981800d-b1b3-4d1d-9445-fd0728e02cc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:11.584936 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:51:11.606135 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-8981800d-b1b3-4d1d-9445-fd0728e02cc9 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:51:11.608683 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-8981800d-b1b3-4d1d-9445-fd0728e02cc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:51:11.718745 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1287/2553] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:51:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:51:11.732403 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6909da-e857-446c-a64c-ff38ae62528c admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:11.733908 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-bb6909da-e857-446c-a64c-ff38ae62528c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:51:11.809188 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6909da-e857-446c-a64c-ff38ae62528c admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:11.813073 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e341506-4954-4037-9c52-0b83f07a938f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:11.813963 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e341506-4954-4037-9c52-0b83f07a938f service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:11.814081 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e341506-4954-4037-9c52-0b83f07a938f service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:11.814273 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e341506-4954-4037-9c52-0b83f07a938f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:11.828442 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e341506-4954-4037-9c52-0b83f07a938f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:11.830238 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e341506-4954-4037-9c52-0b83f07a938f service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:11.842859 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1267/2554] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:51:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:51:12.012895 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e63f110-deed-4fdc-991a-ed7b8e5c7fff None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:12.013458 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e63f110-deed-4fdc-991a-ed7b8e5c7fff None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:12.013744 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e63f110-deed-4fdc-991a-ed7b8e5c7fff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:12.046434 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-5e63f110-deed-4fdc-991a-ed7b8e5c7fff None None] MFA Rules not processed for user `c4bffefe8fb2475a96ea033d07c790f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:51:12.048044 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-5e63f110-deed-4fdc-991a-ed7b8e5c7fff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:51:12.182160 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1288/2555] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:51:12 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:51:12.189492 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e341506-4954-4037-9c52-0b83f07a938f service neutron] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:12.191532 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-0e341506-4954-4037-9c52-0b83f07a938f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:51:12.270831 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e341506-4954-4037-9c52-0b83f07a938f service neutron] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:12.274478 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c053742-ba02-43d3-9309-04e1e3f6ccc5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24d5e2637b7043e08313909d5b06e329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:12.275131 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c053742-ba02-43d3-9309-04e1e3f6ccc5 service cinder] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:12.275418 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c053742-ba02-43d3-9309-04e1e3f6ccc5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:12.275696 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c053742-ba02-43d3-9309-04e1e3f6ccc5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:12.286091 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c053742-ba02-43d3-9309-04e1e3f6ccc5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:12.287540 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c053742-ba02-43d3-9309-04e1e3f6ccc5 service cinder] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:12.297697 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1268/2556] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:51:12 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:51:12.321134 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e63f110-deed-4fdc-991a-ed7b8e5c7fff None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:12.323275 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-5e63f110-deed-4fdc-991a-ed7b8e5c7fff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:51:12.340906 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e63f110-deed-4fdc-991a-ed7b8e5c7fff None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:12.346866 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21a3f67-fded-4896-9839-614815283c3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:12.349020 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21a3f67-fded-4896-9839-614815283c3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:12.349643 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21a3f67-fded-4896-9839-614815283c3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:12.349901 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21a3f67-fded-4896-9839-614815283c3e admin admin] PATH_INFO: `/v3/projects/da66c07c22de41d6ae3a5f5ddcb47af5` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:12.351794 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21a3f67-fded-4896-9839-614815283c3e admin admin] RBAC: Authorizing `identity:delete_project(project_id=da66c07c22de41d6ae3a5f5ddcb47af5)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:12.354600 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21a3f67-fded-4896-9839-614815283c3e admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:12.408768 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-c21a3f67-fded-4896-9839-614815283c3e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da66c07c22de41d6ae3a5f5ddcb47af5', 'request_id': 'req-c21a3f67-fded-4896-9839-614815283c3e'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:12.417646 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-c21a3f67-fded-4896-9839-614815283c3e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da66c07c22de41d6ae3a5f5ddcb47af5', 'request_id': 'req-c21a3f67-fded-4896-9839-614815283c3e'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:12.424140 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-c21a3f67-fded-4896-9839-614815283c3e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da66c07c22de41d6ae3a5f5ddcb47af5', 'request_id': 'req-c21a3f67-fded-4896-9839-614815283c3e'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:12.436115 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-c21a3f67-fded-4896-9839-614815283c3e admin admin] The token cache is being invalidate because project da66c07c22de41d6ae3a5f5ddcb47af5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:51:12.436330 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-c21a3f67-fded-4896-9839-614815283c3e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:12.437019 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-c21a3f67-fded-4896-9839-614815283c3e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:12.439341 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1289/2557] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jun 16 17:51:12 2022] DELETE /identity/v3/projects/da66c07c22de41d6ae3a5f5ddcb47af5 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:51:12.444717 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f0ef21-7ebc-496d-8357-d6fb74d124bc None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:12.445006 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f0ef21-7ebc-496d-8357-d6fb74d124bc None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:12.445276 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f0ef21-7ebc-496d-8357-d6fb74d124bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:12.468180 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-82f0ef21-7ebc-496d-8357-d6fb74d124bc None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:51:12.469593 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-82f0ef21-7ebc-496d-8357-d6fb74d124bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:51:12.484486 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:51:12.541813 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1269/2558] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:51:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:51:12.549822 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eca4562-926a-484e-b303-57010ac9a2ce None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:12.550040 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eca4562-926a-484e-b303-57010ac9a2ce None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:12.550293 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eca4562-926a-484e-b303-57010ac9a2ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:12.579019 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-9eca4562-926a-484e-b303-57010ac9a2ce None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:51:12.580244 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-9eca4562-926a-484e-b303-57010ac9a2ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:51:12.588928 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1290/2559] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:51:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:51:12.593341 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f0ef21-7ebc-496d-8357-d6fb74d124bc None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:12.606710 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f0ef21-7ebc-496d-8357-d6fb74d124bc None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:12.618740 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8a376b-7a87-4fe5-b9af-0ea09613ce05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:12.618740 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8a376b-7a87-4fe5-b9af-0ea09613ce05 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:12.618740 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8a376b-7a87-4fe5-b9af-0ea09613ce05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:12.618740 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8a376b-7a87-4fe5-b9af-0ea09613ce05 admin admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:12.619391 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8a376b-7a87-4fe5-b9af-0ea09613ce05 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:51:12.619391 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8a376b-7a87-4fe5-b9af-0ea09613ce05 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:12.619391 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8a376b-7a87-4fe5-b9af-0ea09613ce05 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:12.630298 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1270/2560] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jun 16 17:51:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:51:12.634664 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eca4562-926a-484e-b303-57010ac9a2ce None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:12.645546 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eca4562-926a-484e-b303-57010ac9a2ce None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:12.657536 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb4a0c7-4953-4224-8ad3-8e5f0fb68719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:12.658207 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb4a0c7-4953-4224-8ad3-8e5f0fb68719 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:12.658490 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb4a0c7-4953-4224-8ad3-8e5f0fb68719 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:12.658780 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb4a0c7-4953-4224-8ad3-8e5f0fb68719 admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:12.659472 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb4a0c7-4953-4224-8ad3-8e5f0fb68719 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:12.662306 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb4a0c7-4953-4224-8ad3-8e5f0fb68719 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:12.671076 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:51:12.690028 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1291/2561] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jun 16 17:51:12 2022] POST /identity/v3/projects => generated 381 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:51:12.695283 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8a376b-7a87-4fe5-b9af-0ea09613ce05 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:12.705386 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8a376b-7a87-4fe5-b9af-0ea09613ce05 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:12.762572 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f8d5f2-b9a2-478f-9909-af7d645d79da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:12.763241 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f8d5f2-b9a2-478f-9909-af7d645d79da admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:12.763561 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f8d5f2-b9a2-478f-9909-af7d645d79da admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:12.763838 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f8d5f2-b9a2-478f-9909-af7d645d79da admin admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:12.764558 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f8d5f2-b9a2-478f-9909-af7d645d79da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:12.766476 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f8d5f2-b9a2-478f-9909-af7d645d79da admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:12.786484 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1271/2562] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jun 16 17:51:12 2022] POST /identity/v3/users => generated 291 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:51:12.790582 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb4a0c7-4953-4224-8ad3-8e5f0fb68719 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:12.801545 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb4a0c7-4953-4224-8ad3-8e5f0fb68719 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:12.806455 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0062b084-e914-4895-8ca8-ca601812a22d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:12.807114 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0062b084-e914-4895-8ca8-ca601812a22d admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:12.807398 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0062b084-e914-4895-8ca8-ca601812a22d admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:12.807678 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0062b084-e914-4895-8ca8-ca601812a22d admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:12.808427 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0062b084-e914-4895-8ca8-ca601812a22d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:12.810014 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0062b084-e914-4895-8ca8-ca601812a22d admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:12.820603 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1292/2563] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:51:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:51:12.824367 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f8d5f2-b9a2-478f-9909-af7d645d79da admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:12.835191 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f8d5f2-b9a2-478f-9909-af7d645d79da admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:12.839930 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8039aeab-ed3a-4172-8b80-a29b26910dd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:12.840566 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8039aeab-ed3a-4172-8b80-a29b26910dd0 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:12.840843 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8039aeab-ed3a-4172-8b80-a29b26910dd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:12.841121 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8039aeab-ed3a-4172-8b80-a29b26910dd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:12.841847 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8039aeab-ed3a-4172-8b80-a29b26910dd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:12.843429 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8039aeab-ed3a-4172-8b80-a29b26910dd0 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:12.853623 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1272/2564] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:51:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:51:12.857544 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0062b084-e914-4895-8ca8-ca601812a22d admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:12.868112 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0062b084-e914-4895-8ca8-ca601812a22d admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:12.872832 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffa8dad-9b89-42b0-98a7-e3bb65af0dfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:12.873713 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffa8dad-9b89-42b0-98a7-e3bb65af0dfc admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:12.873999 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffa8dad-9b89-42b0-98a7-e3bb65af0dfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:12.874278 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffa8dad-9b89-42b0-98a7-e3bb65af0dfc admin admin] PATH_INFO: `/v3/projects/55704b6435104c8e958ae6c5bb202f24/users/0824d8df380e477b85690814b0a2a9e3/roles/0be2075946f54c8485f1c275f746478a` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:12.890384 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffa8dad-9b89-42b0-98a7-e3bb65af0dfc admin admin] RBAC: Authorizing `identity:create_grant(project_id=55704b6435104c8e958ae6c5bb202f24, user_id=0824d8df380e477b85690814b0a2a9e3, role_id=0be2075946f54c8485f1c275f746478a)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:12.893060 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffa8dad-9b89-42b0-98a7-e3bb65af0dfc admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:12.905736 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1293/2565] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:51:12 2022] PUT /identity/v3/projects/55704b6435104c8e958ae6c5bb202f24/users/0824d8df380e477b85690814b0a2a9e3/roles/0be2075946f54c8485f1c275f746478a => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:51:12.910935 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08360cd4-bf96-4dde-8cf4-ae1559244d6e None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:12.911221 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08360cd4-bf96-4dde-8cf4-ae1559244d6e None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:12.911501 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08360cd4-bf96-4dde-8cf4-ae1559244d6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:12.930346 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-08360cd4-bf96-4dde-8cf4-ae1559244d6e None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:51:12.931563 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-08360cd4-bf96-4dde-8cf4-ae1559244d6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:51:12.993506 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1273/2566] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:51:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:51:13.003229 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffa8dad-9b89-42b0-98a7-e3bb65af0dfc admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:13.005836 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-fffa8dad-9b89-42b0-98a7-e3bb65af0dfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:51:13.064917 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffa8dad-9b89-42b0-98a7-e3bb65af0dfc admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:13.069661 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1e8c3f-6867-4e7e-b7df-e2772207ed4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:13.070307 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1e8c3f-6867-4e7e-b7df-e2772207ed4e service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:13.070594 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1e8c3f-6867-4e7e-b7df-e2772207ed4e service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:13.070875 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1e8c3f-6867-4e7e-b7df-e2772207ed4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:13.080473 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1e8c3f-6867-4e7e-b7df-e2772207ed4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:13.082004 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1e8c3f-6867-4e7e-b7df-e2772207ed4e service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:13.092210 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1294/2567] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:51:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:51:13.680669 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c90b66-fbef-4835-8d46-200fe92be7dd None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:13.681280 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c90b66-fbef-4835-8d46-200fe92be7dd None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:13.681280 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c90b66-fbef-4835-8d46-200fe92be7dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:13.701134 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-64c90b66-fbef-4835-8d46-200fe92be7dd None None] MFA Rules not processed for user `c4bffefe8fb2475a96ea033d07c790f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:51:13.702344 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-64c90b66-fbef-4835-8d46-200fe92be7dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:51:13.723814 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:51:13.737159 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a6f23c-61db-4b58-92a6-c7bf53539221 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:13.737452 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a6f23c-61db-4b58-92a6-c7bf53539221 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:13.737749 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a6f23c-61db-4b58-92a6-c7bf53539221 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:13.767824 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-f7a6f23c-61db-4b58-92a6-c7bf53539221 None None] MFA Rules not processed for user `0824d8df380e477b85690814b0a2a9e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:51:13.769111 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-f7a6f23c-61db-4b58-92a6-c7bf53539221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:51:13.776561 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1274/2568] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:51:13 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:51:13.856144 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1295/2569] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:51:13 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:51:13.867080 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c90b66-fbef-4835-8d46-200fe92be7dd None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:13.868837 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-64c90b66-fbef-4835-8d46-200fe92be7dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:51:13.934665 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c90b66-fbef-4835-8d46-200fe92be7dd None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:13.938757 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dad49e-4bfc-4136-a134-e84d74ea506c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24d5e2637b7043e08313909d5b06e329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:13.939532 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dad49e-4bfc-4136-a134-e84d74ea506c service cinder] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:13.939922 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dad49e-4bfc-4136-a134-e84d74ea506c service cinder] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:13.940928 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dad49e-4bfc-4136-a134-e84d74ea506c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:13.951662 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dad49e-4bfc-4136-a134-e84d74ea506c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:13.953153 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dad49e-4bfc-4136-a134-e84d74ea506c service cinder] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:13.963362 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1275/2570] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:51:13 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:51:14.031239 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a6f23c-61db-4b58-92a6-c7bf53539221 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:14.032652 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-f7a6f23c-61db-4b58-92a6-c7bf53539221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:51:14.044669 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a6f23c-61db-4b58-92a6-c7bf53539221 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:14.048557 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61980fea-556a-48bb-8b05-431ca66d7c29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:14.049195 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61980fea-556a-48bb-8b05-431ca66d7c29 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:14.049477 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61980fea-556a-48bb-8b05-431ca66d7c29 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:14.049751 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61980fea-556a-48bb-8b05-431ca66d7c29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:14.051986 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-61980fea-556a-48bb-8b05-431ca66d7c29 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:51:14.062960 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61980fea-556a-48bb-8b05-431ca66d7c29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:14.064613 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61980fea-556a-48bb-8b05-431ca66d7c29 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:14.073618 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1296/2571] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:51:14 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 17:51:15.377580 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aaa672-4484-408d-b38e-8554a6c7cb43 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:15.378152 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aaa672-4484-408d-b38e-8554a6c7cb43 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:15.378300 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aaa672-4484-408d-b38e-8554a6c7cb43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:15.398956 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-79aaa672-4484-408d-b38e-8554a6c7cb43 None None] MFA Rules not processed for user `0824d8df380e477b85690814b0a2a9e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:51:15.400398 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-79aaa672-4484-408d-b38e-8554a6c7cb43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:51:15.408452 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1276/2572] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:51:15 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:51:15.828659 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61980fea-556a-48bb-8b05-431ca66d7c29 service neutron] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:15.838169 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61980fea-556a-48bb-8b05-431ca66d7c29 service neutron] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:15.841875 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d617b472-61eb-487d-9411-dc083b6fa936 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:15.842523 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d617b472-61eb-487d-9411-dc083b6fa936 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:15.842813 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d617b472-61eb-487d-9411-dc083b6fa936 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:15.843091 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d617b472-61eb-487d-9411-dc083b6fa936 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:15.844930 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-d617b472-61eb-487d-9411-dc083b6fa936 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:51:15.858913 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d617b472-61eb-487d-9411-dc083b6fa936 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:15.865450 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d617b472-61eb-487d-9411-dc083b6fa936 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:15.876663 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1297/2573] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:51:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:51:20.142513 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aaa672-4484-408d-b38e-8554a6c7cb43 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:20.143989 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-79aaa672-4484-408d-b38e-8554a6c7cb43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:51:20.158128 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aaa672-4484-408d-b38e-8554a6c7cb43 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:20.162661 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ea8547-b01f-49b2-9dc7-c8521dc37df8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:20.163531 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ea8547-b01f-49b2-9dc7-c8521dc37df8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:20.163810 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ea8547-b01f-49b2-9dc7-c8521dc37df8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:20.164083 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ea8547-b01f-49b2-9dc7-c8521dc37df8 admin admin] PATH_INFO: `/v3/users/8fff53cb0d084d9284825488f6843ee4` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:20.165302 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ea8547-b01f-49b2-9dc7-c8521dc37df8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8fff53cb0d084d9284825488f6843ee4)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:20.167219 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ea8547-b01f-49b2-9dc7-c8521dc37df8 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:20.243221 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-a7ea8547-b01f-49b2-9dc7-c8521dc37df8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8fff53cb0d084d9284825488f6843ee4'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:20.249086 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-a7ea8547-b01f-49b2-9dc7-c8521dc37df8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8fff53cb0d084d9284825488f6843ee4'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:20.266773 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-a7ea8547-b01f-49b2-9dc7-c8521dc37df8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8fff53cb0d084d9284825488f6843ee4'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:20.266773 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-a7ea8547-b01f-49b2-9dc7-c8521dc37df8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8fff53cb0d084d9284825488f6843ee4'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:20.294435 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-a7ea8547-b01f-49b2-9dc7-c8521dc37df8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8fff53cb0d084d9284825488f6843ee4'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:20.326284 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1277/2574] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jun 16 17:51:20 2022] DELETE /identity/v3/users/8fff53cb0d084d9284825488f6843ee4 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:51:20.577928 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d617b472-61eb-487d-9411-dc083b6fa936 service neutron] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:20.592877 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d617b472-61eb-487d-9411-dc083b6fa936 service neutron] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:20.629876 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:51:20.674057 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7284c3cf-8c21-44ff-acb5-6b6cfeb5de06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:20.674925 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7284c3cf-8c21-44ff-acb5-6b6cfeb5de06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:20.675246 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7284c3cf-8c21-44ff-acb5-6b6cfeb5de06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:20.676218 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7284c3cf-8c21-44ff-acb5-6b6cfeb5de06 admin admin] PATH_INFO: `/v3/projects/4cf380fb2e904569b24c6212d88ac621` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:20.677535 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7284c3cf-8c21-44ff-acb5-6b6cfeb5de06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4cf380fb2e904569b24c6212d88ac621)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:20.679288 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7284c3cf-8c21-44ff-acb5-6b6cfeb5de06 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:20.727744 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-7284c3cf-8c21-44ff-acb5-6b6cfeb5de06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4cf380fb2e904569b24c6212d88ac621', 'request_id': 'req-7284c3cf-8c21-44ff-acb5-6b6cfeb5de06'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:20.738030 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-7284c3cf-8c21-44ff-acb5-6b6cfeb5de06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4cf380fb2e904569b24c6212d88ac621', 'request_id': 'req-7284c3cf-8c21-44ff-acb5-6b6cfeb5de06'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:20.746888 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-7284c3cf-8c21-44ff-acb5-6b6cfeb5de06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4cf380fb2e904569b24c6212d88ac621', 'request_id': 'req-7284c3cf-8c21-44ff-acb5-6b6cfeb5de06'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:20.760863 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-7284c3cf-8c21-44ff-acb5-6b6cfeb5de06 admin admin] The token cache is being invalidate because project 4cf380fb2e904569b24c6212d88ac621 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:51:20.761158 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-7284c3cf-8c21-44ff-acb5-6b6cfeb5de06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:20.762656 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-7284c3cf-8c21-44ff-acb5-6b6cfeb5de06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:20.767687 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1298/2575] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jun 16 17:51:20 2022] DELETE /identity/v3/projects/4cf380fb2e904569b24c6212d88ac621 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:51:20.769539 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fc6d12-6719-4180-a352-1a469c9e34ea None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:20.769828 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fc6d12-6719-4180-a352-1a469c9e34ea None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:20.770109 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fc6d12-6719-4180-a352-1a469c9e34ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:20.789167 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-d8fc6d12-6719-4180-a352-1a469c9e34ea None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:51:20.790460 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-d8fc6d12-6719-4180-a352-1a469c9e34ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:51:20.832349 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:51:20.866889 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1278/2576] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:51:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:51:20.873950 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8848796-bcc5-4ffc-92af-e2fb21e51712 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:20.874255 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8848796-bcc5-4ffc-92af-e2fb21e51712 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:20.874704 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8848796-bcc5-4ffc-92af-e2fb21e51712 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:20.903211 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-e8848796-bcc5-4ffc-92af-e2fb21e51712 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:51:20.904488 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-e8848796-bcc5-4ffc-92af-e2fb21e51712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:51:20.913215 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1299/2577] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:51:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:51:20.917571 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fc6d12-6719-4180-a352-1a469c9e34ea None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:20.930498 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fc6d12-6719-4180-a352-1a469c9e34ea None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:20.935122 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebf8030-e24e-4adf-a3c4-9d737caa5b87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:20.935781 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebf8030-e24e-4adf-a3c4-9d737caa5b87 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:20.936063 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebf8030-e24e-4adf-a3c4-9d737caa5b87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:20.936338 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebf8030-e24e-4adf-a3c4-9d737caa5b87 admin admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:20.937094 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebf8030-e24e-4adf-a3c4-9d737caa5b87 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:51:20.937458 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebf8030-e24e-4adf-a3c4-9d737caa5b87 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:20.939184 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebf8030-e24e-4adf-a3c4-9d737caa5b87 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:20.950839 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1279/2578] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jun 16 17:51:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:51:20.955115 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8848796-bcc5-4ffc-92af-e2fb21e51712 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:20.965158 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8848796-bcc5-4ffc-92af-e2fb21e51712 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:20.969834 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7ebb77-6435-4877-91c6-76a900cd48ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:20.970480 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7ebb77-6435-4877-91c6-76a900cd48ec admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:20.970763 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7ebb77-6435-4877-91c6-76a900cd48ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:20.971039 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7ebb77-6435-4877-91c6-76a900cd48ec admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:20.971722 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7ebb77-6435-4877-91c6-76a900cd48ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:20.974726 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7ebb77-6435-4877-91c6-76a900cd48ec admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:21.000844 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1300/2579] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jun 16 17:51:20 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:51:21.005067 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebf8030-e24e-4adf-a3c4-9d737caa5b87 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:21.015423 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebf8030-e24e-4adf-a3c4-9d737caa5b87 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:21.073283 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e598f3-ae7c-40e9-a5ea-d3c9c757699d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:21.073947 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e598f3-ae7c-40e9-a5ea-d3c9c757699d admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:21.074233 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e598f3-ae7c-40e9-a5ea-d3c9c757699d admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:21.074507 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e598f3-ae7c-40e9-a5ea-d3c9c757699d admin admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:21.075183 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e598f3-ae7c-40e9-a5ea-d3c9c757699d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:21.077188 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e598f3-ae7c-40e9-a5ea-d3c9c757699d admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:21.097018 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1280/2580] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jun 16 17:51:21 2022] POST /identity/v3/users => generated 287 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:51:21.101682 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7ebb77-6435-4877-91c6-76a900cd48ec admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:21.112508 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7ebb77-6435-4877-91c6-76a900cd48ec admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:21.118040 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296f20c4-d30c-40b2-bed6-dd54fbdcaa72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:21.118744 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296f20c4-d30c-40b2-bed6-dd54fbdcaa72 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:21.119030 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296f20c4-d30c-40b2-bed6-dd54fbdcaa72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:21.119314 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296f20c4-d30c-40b2-bed6-dd54fbdcaa72 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:21.120048 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296f20c4-d30c-40b2-bed6-dd54fbdcaa72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:21.121748 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296f20c4-d30c-40b2-bed6-dd54fbdcaa72 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:21.133446 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1301/2581] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:51:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:51:21.137591 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e598f3-ae7c-40e9-a5ea-d3c9c757699d admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:21.147957 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e598f3-ae7c-40e9-a5ea-d3c9c757699d admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:21.152889 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabfa549-4911-4612-a95e-eba022051727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:21.153555 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabfa549-4911-4612-a95e-eba022051727 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:21.153845 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabfa549-4911-4612-a95e-eba022051727 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:21.154125 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabfa549-4911-4612-a95e-eba022051727 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:21.154851 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabfa549-4911-4612-a95e-eba022051727 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:21.156482 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabfa549-4911-4612-a95e-eba022051727 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:21.167418 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1281/2582] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:51:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:51:21.171535 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296f20c4-d30c-40b2-bed6-dd54fbdcaa72 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:21.182459 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296f20c4-d30c-40b2-bed6-dd54fbdcaa72 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:21.187251 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f0b0b5-fe05-4388-bac4-c14ab5dd0c67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:21.188187 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f0b0b5-fe05-4388-bac4-c14ab5dd0c67 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:21.188493 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f0b0b5-fe05-4388-bac4-c14ab5dd0c67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:21.188770 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f0b0b5-fe05-4388-bac4-c14ab5dd0c67 admin admin] PATH_INFO: `/v3/projects/151a95de1f164fb290d59ed9fdf70d14/users/ea0166b3c740455784ec673c20adbd7f/roles/0be2075946f54c8485f1c275f746478a` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:21.203937 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f0b0b5-fe05-4388-bac4-c14ab5dd0c67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=151a95de1f164fb290d59ed9fdf70d14, user_id=ea0166b3c740455784ec673c20adbd7f, role_id=0be2075946f54c8485f1c275f746478a)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:21.206640 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f0b0b5-fe05-4388-bac4-c14ab5dd0c67 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:21.219693 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1302/2583] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:51:21 2022] PUT /identity/v3/projects/151a95de1f164fb290d59ed9fdf70d14/users/ea0166b3c740455784ec673c20adbd7f/roles/0be2075946f54c8485f1c275f746478a => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:51:21.229762 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a74dab5-94aa-47f1-ba7b-83fd5f78ce9b None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:21.230258 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a74dab5-94aa-47f1-ba7b-83fd5f78ce9b None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:21.230809 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a74dab5-94aa-47f1-ba7b-83fd5f78ce9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:21.258894 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-9a74dab5-94aa-47f1-ba7b-83fd5f78ce9b None None] MFA Rules not processed for user `ea0166b3c740455784ec673c20adbd7f`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:51:21.260337 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-9a74dab5-94aa-47f1-ba7b-83fd5f78ce9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:51:21.379947 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1282/2584] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:51:21 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:51:21.521263 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f0b0b5-fe05-4388-bac4-c14ab5dd0c67 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:21.522997 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-19f0b0b5-fe05-4388-bac4-c14ab5dd0c67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:51:21.575134 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:51:21.601774 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f0b0b5-fe05-4388-bac4-c14ab5dd0c67 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:21.607182 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f96b26e-29b6-4800-8438-19363bf7ced7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f86c16179a70474a817be9fda33ece9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:21.607823 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f96b26e-29b6-4800-8438-19363bf7ced7 service nova] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:21.608102 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f96b26e-29b6-4800-8438-19363bf7ced7 service nova] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:21.608388 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f96b26e-29b6-4800-8438-19363bf7ced7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:21.616694 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f96b26e-29b6-4800-8438-19363bf7ced7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:21.618252 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f96b26e-29b6-4800-8438-19363bf7ced7 service nova] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:21.626944 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1303/2585] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:51:21 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:51:24.649959 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a74dab5-94aa-47f1-ba7b-83fd5f78ce9b None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:24.686651 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:51:24.719608 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a74dab5-94aa-47f1-ba7b-83fd5f78ce9b None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:24.726617 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0a84ff-1add-4795-bbd9-8affb0c33343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:24.727539 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0a84ff-1add-4795-bbd9-8affb0c33343 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:24.727877 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0a84ff-1add-4795-bbd9-8affb0c33343 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:24.728040 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0a84ff-1add-4795-bbd9-8affb0c33343 admin admin] PATH_INFO: `/v3/users/ea0166b3c740455784ec673c20adbd7f` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:24.729264 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0a84ff-1add-4795-bbd9-8affb0c33343 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea0166b3c740455784ec673c20adbd7f)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:24.731193 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0a84ff-1add-4795-bbd9-8affb0c33343 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:24.795956 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-7b0a84ff-1add-4795-bbd9-8affb0c33343 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea0166b3c740455784ec673c20adbd7f'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:24.796736 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-7b0a84ff-1add-4795-bbd9-8affb0c33343 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea0166b3c740455784ec673c20adbd7f'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:24.806648 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-7b0a84ff-1add-4795-bbd9-8affb0c33343 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea0166b3c740455784ec673c20adbd7f'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:24.807567 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-7b0a84ff-1add-4795-bbd9-8affb0c33343 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea0166b3c740455784ec673c20adbd7f'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:24.824673 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-7b0a84ff-1add-4795-bbd9-8affb0c33343 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea0166b3c740455784ec673c20adbd7f'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:24.838354 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1283/2586] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jun 16 17:51:24 2022] DELETE /identity/v3/users/ea0166b3c740455784ec673c20adbd7f => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:51:24.843152 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b6295f-e6cc-4212-b530-2c8689d40ed4 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:24.843664 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b6295f-e6cc-4212-b530-2c8689d40ed4 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:24.843750 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b6295f-e6cc-4212-b530-2c8689d40ed4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:24.861999 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-c9b6295f-e6cc-4212-b530-2c8689d40ed4 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:51:24.863819 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-c9b6295f-e6cc-4212-b530-2c8689d40ed4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:51:24.930821 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1304/2587] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:51:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:51:24.940476 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0a84ff-1add-4795-bbd9-8affb0c33343 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:24.942564 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-7b0a84ff-1add-4795-bbd9-8affb0c33343 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:51:25.005079 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0a84ff-1add-4795-bbd9-8affb0c33343 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:25.008984 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f50c0f-47a3-4640-9120-5ffaa7c356bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:25.009648 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f50c0f-47a3-4640-9120-5ffaa7c356bf service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:25.009933 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f50c0f-47a3-4640-9120-5ffaa7c356bf service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:25.010212 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f50c0f-47a3-4640-9120-5ffaa7c356bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:25.026366 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f50c0f-47a3-4640-9120-5ffaa7c356bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:25.027825 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f50c0f-47a3-4640-9120-5ffaa7c356bf service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:25.039955 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1284/2588] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:51:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:51:25.338916 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b6295f-e6cc-4212-b530-2c8689d40ed4 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:25.340400 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-c9b6295f-e6cc-4212-b530-2c8689d40ed4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:51:25.353954 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b6295f-e6cc-4212-b530-2c8689d40ed4 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:25.358720 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9f5a3d-6507-49ba-9319-95e6623faaf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:25.359583 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9f5a3d-6507-49ba-9319-95e6623faaf3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:25.359867 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9f5a3d-6507-49ba-9319-95e6623faaf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:25.360146 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9f5a3d-6507-49ba-9319-95e6623faaf3 admin admin] PATH_INFO: `/v3/projects/151a95de1f164fb290d59ed9fdf70d14` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:25.361393 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9f5a3d-6507-49ba-9319-95e6623faaf3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=151a95de1f164fb290d59ed9fdf70d14)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:25.364154 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9f5a3d-6507-49ba-9319-95e6623faaf3 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:25.415918 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-2e9f5a3d-6507-49ba-9319-95e6623faaf3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '151a95de1f164fb290d59ed9fdf70d14', 'request_id': 'req-2e9f5a3d-6507-49ba-9319-95e6623faaf3'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:25.422979 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-2e9f5a3d-6507-49ba-9319-95e6623faaf3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '151a95de1f164fb290d59ed9fdf70d14', 'request_id': 'req-2e9f5a3d-6507-49ba-9319-95e6623faaf3'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:25.430282 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-2e9f5a3d-6507-49ba-9319-95e6623faaf3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '151a95de1f164fb290d59ed9fdf70d14', 'request_id': 'req-2e9f5a3d-6507-49ba-9319-95e6623faaf3'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:25.441241 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-2e9f5a3d-6507-49ba-9319-95e6623faaf3 admin admin] The token cache is being invalidate because project 151a95de1f164fb290d59ed9fdf70d14 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:51:25.441765 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-2e9f5a3d-6507-49ba-9319-95e6623faaf3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:25.442514 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-2e9f5a3d-6507-49ba-9319-95e6623faaf3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:25.444195 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1305/2589] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jun 16 17:51:25 2022] DELETE /identity/v3/projects/151a95de1f164fb290d59ed9fdf70d14 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:51:25.450177 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a975c272-69a2-46fd-8733-8905c3d43a61 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:25.450734 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a975c272-69a2-46fd-8733-8905c3d43a61 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:25.450909 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a975c272-69a2-46fd-8733-8905c3d43a61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:25.469667 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-a975c272-69a2-46fd-8733-8905c3d43a61 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:51:25.470992 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-a975c272-69a2-46fd-8733-8905c3d43a61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:51:25.527222 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:51:25.544618 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1285/2590] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:51:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:51:25.552995 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f157ab-c5ec-4760-a3be-fe8eeff73b4f None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:25.554000 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f157ab-c5ec-4760-a3be-fe8eeff73b4f None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:25.554353 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f157ab-c5ec-4760-a3be-fe8eeff73b4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:25.574346 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-99f157ab-c5ec-4760-a3be-fe8eeff73b4f None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:51:25.575755 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-99f157ab-c5ec-4760-a3be-fe8eeff73b4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:51:25.584348 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1306/2591] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:51:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:51:25.588633 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a975c272-69a2-46fd-8733-8905c3d43a61 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:25.599043 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a975c272-69a2-46fd-8733-8905c3d43a61 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:25.603963 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5d3e9b-eaff-482c-9c9a-491534a8421e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:25.604683 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5d3e9b-eaff-482c-9c9a-491534a8421e admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:25.604967 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5d3e9b-eaff-482c-9c9a-491534a8421e admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:25.605432 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5d3e9b-eaff-482c-9c9a-491534a8421e admin admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:25.606567 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5d3e9b-eaff-482c-9c9a-491534a8421e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:51:25.606886 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5d3e9b-eaff-482c-9c9a-491534a8421e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:25.608819 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5d3e9b-eaff-482c-9c9a-491534a8421e admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:25.624671 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1286/2592] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jun 16 17:51:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:51:25.629781 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f157ab-c5ec-4760-a3be-fe8eeff73b4f None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:25.640094 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f157ab-c5ec-4760-a3be-fe8eeff73b4f None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:25.645334 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b9d6de-e8c5-4a8a-b9e2-462df4ecf9d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:25.646177 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b9d6de-e8c5-4a8a-b9e2-462df4ecf9d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:25.646586 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b9d6de-e8c5-4a8a-b9e2-462df4ecf9d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:25.646958 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b9d6de-e8c5-4a8a-b9e2-462df4ecf9d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:25.647748 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b9d6de-e8c5-4a8a-b9e2-462df4ecf9d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:25.649777 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b9d6de-e8c5-4a8a-b9e2-462df4ecf9d8 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:25.677809 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1307/2593] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jun 16 17:51:25 2022] POST /identity/v3/projects => generated 365 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:51:25.681969 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5d3e9b-eaff-482c-9c9a-491534a8421e admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:25.693780 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5d3e9b-eaff-482c-9c9a-491534a8421e admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:25.758350 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8e79ce-f937-4e61-82b8-0bbb5796122f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:25.758951 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8e79ce-f937-4e61-82b8-0bbb5796122f admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:25.759239 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8e79ce-f937-4e61-82b8-0bbb5796122f admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:25.759519 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8e79ce-f937-4e61-82b8-0bbb5796122f admin admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:25.760215 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8e79ce-f937-4e61-82b8-0bbb5796122f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:25.762447 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8e79ce-f937-4e61-82b8-0bbb5796122f admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:25.788833 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1287/2594] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jun 16 17:51:25 2022] POST /identity/v3/users => generated 283 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:51:25.790039 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b9d6de-e8c5-4a8a-b9e2-462df4ecf9d8 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:25.800739 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b9d6de-e8c5-4a8a-b9e2-462df4ecf9d8 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:25.805571 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7fff95-374f-4d28-9a20-0c6d0a9cc3b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:25.806236 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7fff95-374f-4d28-9a20-0c6d0a9cc3b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:25.806520 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7fff95-374f-4d28-9a20-0c6d0a9cc3b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:25.806795 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7fff95-374f-4d28-9a20-0c6d0a9cc3b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:25.807521 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7fff95-374f-4d28-9a20-0c6d0a9cc3b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:25.809244 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7fff95-374f-4d28-9a20-0c6d0a9cc3b2 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:25.820131 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1308/2595] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:51:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:51:25.824140 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8e79ce-f937-4e61-82b8-0bbb5796122f admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:25.835141 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8e79ce-f937-4e61-82b8-0bbb5796122f admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:25.840010 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213d087a-9fea-4d4b-b3e5-74381c5fd564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:25.840807 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213d087a-9fea-4d4b-b3e5-74381c5fd564 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:25.841095 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213d087a-9fea-4d4b-b3e5-74381c5fd564 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:25.841612 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213d087a-9fea-4d4b-b3e5-74381c5fd564 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:25.842474 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213d087a-9fea-4d4b-b3e5-74381c5fd564 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:25.844084 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213d087a-9fea-4d4b-b3e5-74381c5fd564 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:25.855284 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1288/2596] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:51:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:51:25.859448 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7fff95-374f-4d28-9a20-0c6d0a9cc3b2 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:25.871483 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7fff95-374f-4d28-9a20-0c6d0a9cc3b2 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:25.876590 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088b7e5f-54ab-4d29-adfc-6aac0110a061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:25.877604 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088b7e5f-54ab-4d29-adfc-6aac0110a061 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:25.877913 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088b7e5f-54ab-4d29-adfc-6aac0110a061 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:25.878189 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088b7e5f-54ab-4d29-adfc-6aac0110a061 admin admin] PATH_INFO: `/v3/projects/7155871877b243ecb7aaff98172d1124/users/c7e3a98d3bde4b4c8abb4c6d147c3425/roles/0be2075946f54c8485f1c275f746478a` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:25.896952 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:51:25.900515 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088b7e5f-54ab-4d29-adfc-6aac0110a061 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7155871877b243ecb7aaff98172d1124, user_id=c7e3a98d3bde4b4c8abb4c6d147c3425, role_id=0be2075946f54c8485f1c275f746478a)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:25.905743 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088b7e5f-54ab-4d29-adfc-6aac0110a061 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:25.921693 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1309/2597] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:51:25 2022] PUT /identity/v3/projects/7155871877b243ecb7aaff98172d1124/users/c7e3a98d3bde4b4c8abb4c6d147c3425/roles/0be2075946f54c8485f1c275f746478a => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:51:25.932461 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70056a91-c5e3-40b0-a935-5ecf58039474 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:25.932768 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70056a91-c5e3-40b0-a935-5ecf58039474 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:25.933058 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70056a91-c5e3-40b0-a935-5ecf58039474 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:25.978256 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-70056a91-c5e3-40b0-a935-5ecf58039474 None None] MFA Rules not processed for user `c7e3a98d3bde4b4c8abb4c6d147c3425`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:51:25.979481 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-70056a91-c5e3-40b0-a935-5ecf58039474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:51:26.048211 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1289/2598] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:51:25 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:51:26.053539 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088b7e5f-54ab-4d29-adfc-6aac0110a061 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:26.114195 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088b7e5f-54ab-4d29-adfc-6aac0110a061 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:26.118888 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5d3701-bb24-4cf5-806b-690eb8bc2b11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:26.119538 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5d3701-bb24-4cf5-806b-690eb8bc2b11 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:26.119822 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5d3701-bb24-4cf5-806b-690eb8bc2b11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:26.120098 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5d3701-bb24-4cf5-806b-690eb8bc2b11 admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:26.120836 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5d3701-bb24-4cf5-806b-690eb8bc2b11 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:26.122723 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5d3701-bb24-4cf5-806b-690eb8bc2b11 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:26.146995 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1310/2599] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jun 16 17:51:26 2022] POST /identity/v3/projects => generated 365 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:51:26.151127 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70056a91-c5e3-40b0-a935-5ecf58039474 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:26.163192 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70056a91-c5e3-40b0-a935-5ecf58039474 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:26.211279 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:51:26.223041 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cd6dd6-1ff1-4942-bba2-3b6d73b3693c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:26.223751 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cd6dd6-1ff1-4942-bba2-3b6d73b3693c admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:26.224036 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cd6dd6-1ff1-4942-bba2-3b6d73b3693c admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:26.224314 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cd6dd6-1ff1-4942-bba2-3b6d73b3693c admin admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:26.225082 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cd6dd6-1ff1-4942-bba2-3b6d73b3693c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:26.227011 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cd6dd6-1ff1-4942-bba2-3b6d73b3693c admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:26.246672 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1290/2600] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jun 16 17:51:26 2022] POST /identity/v3/users => generated 289 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:51:26.250437 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5d3701-bb24-4cf5-806b-690eb8bc2b11 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:26.261529 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5d3701-bb24-4cf5-806b-690eb8bc2b11 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:26.266452 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0109fa17-c5af-4f6f-9862-4f75baa8ca99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:26.267112 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0109fa17-c5af-4f6f-9862-4f75baa8ca99 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:26.267396 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0109fa17-c5af-4f6f-9862-4f75baa8ca99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:26.267675 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0109fa17-c5af-4f6f-9862-4f75baa8ca99 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:26.268417 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0109fa17-c5af-4f6f-9862-4f75baa8ca99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:26.270572 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0109fa17-c5af-4f6f-9862-4f75baa8ca99 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:26.281046 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1311/2601] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:51:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:51:26.285428 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cd6dd6-1ff1-4942-bba2-3b6d73b3693c admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:26.295485 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cd6dd6-1ff1-4942-bba2-3b6d73b3693c admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:26.300150 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fc0dbf-23fb-491d-98c6-f091644dbf50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:26.301725 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fc0dbf-23fb-491d-98c6-f091644dbf50 admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:26.302012 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fc0dbf-23fb-491d-98c6-f091644dbf50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:26.302293 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fc0dbf-23fb-491d-98c6-f091644dbf50 admin admin] PATH_INFO: `/v3/projects/3927fa7149a74c4da6a0b80d20fc6886/users/4c6ef49921ed45078a61c1c7928423c3/roles/3ca2dcbbd29d4a31a3ab78f2299bc239` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:26.318730 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fc0dbf-23fb-491d-98c6-f091644dbf50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3927fa7149a74c4da6a0b80d20fc6886, user_id=4c6ef49921ed45078a61c1c7928423c3, role_id=3ca2dcbbd29d4a31a3ab78f2299bc239)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:26.321604 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fc0dbf-23fb-491d-98c6-f091644dbf50 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:26.336638 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1291/2602] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:51:26 2022] PUT /identity/v3/projects/3927fa7149a74c4da6a0b80d20fc6886/users/4c6ef49921ed45078a61c1c7928423c3/roles/3ca2dcbbd29d4a31a3ab78f2299bc239 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:51:26.340654 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0109fa17-c5af-4f6f-9862-4f75baa8ca99 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:26.351806 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0109fa17-c5af-4f6f-9862-4f75baa8ca99 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:26.409764 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b28e090-3159-45f1-99ea-dca28700b1ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:26.411029 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b28e090-3159-45f1-99ea-dca28700b1ed admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:26.411314 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b28e090-3159-45f1-99ea-dca28700b1ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:26.411594 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b28e090-3159-45f1-99ea-dca28700b1ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:26.412889 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b28e090-3159-45f1-99ea-dca28700b1ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:26.414820 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b28e090-3159-45f1-99ea-dca28700b1ed admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:26.431184 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1312/2603] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:51:26 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:51:26.432754 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fc0dbf-23fb-491d-98c6-f091644dbf50 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:26.444586 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fc0dbf-23fb-491d-98c6-f091644dbf50 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:26.449576 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7a6f01-3139-4cf3-9216-4a787f581146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:26.450490 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7a6f01-3139-4cf3-9216-4a787f581146 admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:26.450777 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7a6f01-3139-4cf3-9216-4a787f581146 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:26.451055 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7a6f01-3139-4cf3-9216-4a787f581146 admin admin] PATH_INFO: `/v3/projects/3927fa7149a74c4da6a0b80d20fc6886/users/4c6ef49921ed45078a61c1c7928423c3/roles/3ca2dcbbd29d4a31a3ab78f2299bc239` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:26.453349 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7a6f01-3139-4cf3-9216-4a787f581146 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3927fa7149a74c4da6a0b80d20fc6886, user_id=4c6ef49921ed45078a61c1c7928423c3, role_id=3ca2dcbbd29d4a31a3ab78f2299bc239)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:26.456466 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7a6f01-3139-4cf3-9216-4a787f581146 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:26.467520 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1292/2604] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:51:26 2022] PUT /identity/v3/projects/3927fa7149a74c4da6a0b80d20fc6886/users/4c6ef49921ed45078a61c1c7928423c3/roles/3ca2dcbbd29d4a31a3ab78f2299bc239 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:51:26.478822 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4611b0-954e-41f7-bf4d-12711ddaa37d None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:26.479117 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4611b0-954e-41f7-bf4d-12711ddaa37d None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:26.479406 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4611b0-954e-41f7-bf4d-12711ddaa37d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:26.511133 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-8e4611b0-954e-41f7-bf4d-12711ddaa37d None None] MFA Rules not processed for user `4c6ef49921ed45078a61c1c7928423c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:51:26.512657 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-8e4611b0-954e-41f7-bf4d-12711ddaa37d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:51:26.561033 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:51:26.583847 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1313/2605] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:51:26 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:51:26.595315 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7a6f01-3139-4cf3-9216-4a787f581146 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:26.597239 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-2a7a6f01-3139-4cf3-9216-4a787f581146 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:51:26.662023 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7a6f01-3139-4cf3-9216-4a787f581146 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:26.665685 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eebe19f-3dbe-4a61-a9e3-11c3963d3ea3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24d5e2637b7043e08313909d5b06e329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:26.666308 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eebe19f-3dbe-4a61-a9e3-11c3963d3ea3 service cinder] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:26.666587 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eebe19f-3dbe-4a61-a9e3-11c3963d3ea3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:26.666860 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eebe19f-3dbe-4a61-a9e3-11c3963d3ea3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:26.678011 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eebe19f-3dbe-4a61-a9e3-11c3963d3ea3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:26.679483 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eebe19f-3dbe-4a61-a9e3-11c3963d3ea3 service cinder] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:26.689767 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1293/2606] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:51:26 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:51:27.145410 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4611b0-954e-41f7-bf4d-12711ddaa37d None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:27.261310 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4611b0-954e-41f7-bf4d-12711ddaa37d None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:27.271051 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf9eabb-0c88-4fc4-9b9e-69f3cff8e672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:27.272289 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf9eabb-0c88-4fc4-9b9e-69f3cff8e672 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:27.272760 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf9eabb-0c88-4fc4-9b9e-69f3cff8e672 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:27.273144 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf9eabb-0c88-4fc4-9b9e-69f3cff8e672 admin admin] PATH_INFO: `/v3/users/c7e3a98d3bde4b4c8abb4c6d147c3425` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:27.274502 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf9eabb-0c88-4fc4-9b9e-69f3cff8e672 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7e3a98d3bde4b4c8abb4c6d147c3425)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:27.276840 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf9eabb-0c88-4fc4-9b9e-69f3cff8e672 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:27.370134 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-7cf9eabb-0c88-4fc4-9b9e-69f3cff8e672 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7e3a98d3bde4b4c8abb4c6d147c3425'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:27.389994 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-7cf9eabb-0c88-4fc4-9b9e-69f3cff8e672 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7e3a98d3bde4b4c8abb4c6d147c3425'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:27.391572 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-7cf9eabb-0c88-4fc4-9b9e-69f3cff8e672 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7e3a98d3bde4b4c8abb4c6d147c3425'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:27.404177 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-7cf9eabb-0c88-4fc4-9b9e-69f3cff8e672 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7e3a98d3bde4b4c8abb4c6d147c3425'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:27.414527 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-7cf9eabb-0c88-4fc4-9b9e-69f3cff8e672 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7e3a98d3bde4b4c8abb4c6d147c3425'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:27.422567 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1314/2607] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jun 16 17:51:27 2022] DELETE /identity/v3/users/c7e3a98d3bde4b4c8abb4c6d147c3425 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:51:27.427326 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0344b4e9-3650-431a-b85f-faae11f795c4 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:27.427623 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0344b4e9-3650-431a-b85f-faae11f795c4 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:27.427901 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0344b4e9-3650-431a-b85f-faae11f795c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:27.447292 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-0344b4e9-3650-431a-b85f-faae11f795c4 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:51:27.448568 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-0344b4e9-3650-431a-b85f-faae11f795c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:51:27.528475 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1294/2608] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:51:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:51:27.539015 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf9eabb-0c88-4fc4-9b9e-69f3cff8e672 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:27.541022 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-7cf9eabb-0c88-4fc4-9b9e-69f3cff8e672 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:51:27.608444 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf9eabb-0c88-4fc4-9b9e-69f3cff8e672 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:27.613304 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dcba1a-c9bb-44c8-93e2-230ead809490 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:27.613966 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dcba1a-c9bb-44c8-93e2-230ead809490 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:27.614351 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dcba1a-c9bb-44c8-93e2-230ead809490 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:27.614635 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dcba1a-c9bb-44c8-93e2-230ead809490 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:27.625704 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dcba1a-c9bb-44c8-93e2-230ead809490 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:27.627152 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dcba1a-c9bb-44c8-93e2-230ead809490 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:27.652585 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1315/2609] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:51:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:51:28.030850 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0344b4e9-3650-431a-b85f-faae11f795c4 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:28.032394 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-0344b4e9-3650-431a-b85f-faae11f795c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:51:28.045939 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0344b4e9-3650-431a-b85f-faae11f795c4 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:28.053481 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b27b7b-3302-462f-9f58-ac75aca776db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:28.054418 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b27b7b-3302-462f-9f58-ac75aca776db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:28.054714 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b27b7b-3302-462f-9f58-ac75aca776db admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:28.054988 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b27b7b-3302-462f-9f58-ac75aca776db admin admin] PATH_INFO: `/v3/projects/7155871877b243ecb7aaff98172d1124` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:28.056212 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b27b7b-3302-462f-9f58-ac75aca776db admin admin] RBAC: Authorizing `identity:delete_project(project_id=7155871877b243ecb7aaff98172d1124)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:28.058257 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b27b7b-3302-462f-9f58-ac75aca776db admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:28.101102 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:51:28.108840 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-71b27b7b-3302-462f-9f58-ac75aca776db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7155871877b243ecb7aaff98172d1124', 'request_id': 'req-71b27b7b-3302-462f-9f58-ac75aca776db'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:28.117892 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-71b27b7b-3302-462f-9f58-ac75aca776db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7155871877b243ecb7aaff98172d1124', 'request_id': 'req-71b27b7b-3302-462f-9f58-ac75aca776db'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:28.123919 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-71b27b7b-3302-462f-9f58-ac75aca776db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7155871877b243ecb7aaff98172d1124', 'request_id': 'req-71b27b7b-3302-462f-9f58-ac75aca776db'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:28.136029 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-71b27b7b-3302-462f-9f58-ac75aca776db admin admin] The token cache is being invalidate because project 7155871877b243ecb7aaff98172d1124 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80357) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:51:28.136316 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-71b27b7b-3302-462f-9f58-ac75aca776db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:28.137305 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-71b27b7b-3302-462f-9f58-ac75aca776db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:28.139125 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1295/2610] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jun 16 17:51:28 2022] DELETE /identity/v3/projects/7155871877b243ecb7aaff98172d1124 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:51:28.143042 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dcba1a-c9bb-44c8-93e2-230ead809490 service neutron] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:28.153275 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dcba1a-c9bb-44c8-93e2-230ead809490 service neutron] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:28.158847 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:51:28.225010 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9399659-3bea-45e6-81c5-a872a09229c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:28.225910 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9399659-3bea-45e6-81c5-a872a09229c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:28.226192 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9399659-3bea-45e6-81c5-a872a09229c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:28.226482 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9399659-3bea-45e6-81c5-a872a09229c0 admin admin] PATH_INFO: `/v3/users/4c6ef49921ed45078a61c1c7928423c3` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:28.227664 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9399659-3bea-45e6-81c5-a872a09229c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c6ef49921ed45078a61c1c7928423c3)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:28.229717 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9399659-3bea-45e6-81c5-a872a09229c0 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:28.277307 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-c9399659-3bea-45e6-81c5-a872a09229c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c6ef49921ed45078a61c1c7928423c3'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:28.291849 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-c9399659-3bea-45e6-81c5-a872a09229c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c6ef49921ed45078a61c1c7928423c3'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:28.293053 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-c9399659-3bea-45e6-81c5-a872a09229c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c6ef49921ed45078a61c1c7928423c3'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:28.300188 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-c9399659-3bea-45e6-81c5-a872a09229c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c6ef49921ed45078a61c1c7928423c3'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:28.307923 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-c9399659-3bea-45e6-81c5-a872a09229c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c6ef49921ed45078a61c1c7928423c3'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:28.314064 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1316/2611] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jun 16 17:51:28 2022] DELETE /identity/v3/users/4c6ef49921ed45078a61c1c7928423c3 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:51:28.584194 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b27b7b-3302-462f-9f58-ac75aca776db admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:28.585792 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-71b27b7b-3302-462f-9f58-ac75aca776db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:51:28.601598 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b27b7b-3302-462f-9f58-ac75aca776db admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:28.659827 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22540dd3-f7b3-4a98-8d07-9a4b81e67d46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:28.661127 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22540dd3-f7b3-4a98-8d07-9a4b81e67d46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:28.661414 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22540dd3-f7b3-4a98-8d07-9a4b81e67d46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:28.661693 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22540dd3-f7b3-4a98-8d07-9a4b81e67d46 admin admin] PATH_INFO: `/v3/projects/3927fa7149a74c4da6a0b80d20fc6886` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:28.662883 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22540dd3-f7b3-4a98-8d07-9a4b81e67d46 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3927fa7149a74c4da6a0b80d20fc6886)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:28.664846 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22540dd3-f7b3-4a98-8d07-9a4b81e67d46 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:28.711724 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-22540dd3-f7b3-4a98-8d07-9a4b81e67d46 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3927fa7149a74c4da6a0b80d20fc6886', 'request_id': 'req-22540dd3-f7b3-4a98-8d07-9a4b81e67d46'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:28.719324 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-22540dd3-f7b3-4a98-8d07-9a4b81e67d46 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3927fa7149a74c4da6a0b80d20fc6886', 'request_id': 'req-22540dd3-f7b3-4a98-8d07-9a4b81e67d46'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:28.725239 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-22540dd3-f7b3-4a98-8d07-9a4b81e67d46 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3927fa7149a74c4da6a0b80d20fc6886', 'request_id': 'req-22540dd3-f7b3-4a98-8d07-9a4b81e67d46'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:28.736329 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-22540dd3-f7b3-4a98-8d07-9a4b81e67d46 admin admin] The token cache is being invalidate because project 3927fa7149a74c4da6a0b80d20fc6886 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80357) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:51:28.736687 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-22540dd3-f7b3-4a98-8d07-9a4b81e67d46 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:28.737342 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-22540dd3-f7b3-4a98-8d07-9a4b81e67d46 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:28.739555 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1296/2612] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jun 16 17:51:28 2022] DELETE /identity/v3/projects/3927fa7149a74c4da6a0b80d20fc6886 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:51:28.745341 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bd98ad-f9b3-4817-bcf1-3c42f86862ec None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:28.745638 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bd98ad-f9b3-4817-bcf1-3c42f86862ec None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:28.745922 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bd98ad-f9b3-4817-bcf1-3c42f86862ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:28.765152 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-b9bd98ad-f9b3-4817-bcf1-3c42f86862ec None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:51:28.766348 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-b9bd98ad-f9b3-4817-bcf1-3c42f86862ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:51:28.824321 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1317/2613] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:51:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:51:28.834001 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c528e8-4339-4e04-88cb-54fbe644b5fb None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:28.834294 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c528e8-4339-4e04-88cb-54fbe644b5fb None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:28.834578 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c528e8-4339-4e04-88cb-54fbe644b5fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:28.853428 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-e4c528e8-4339-4e04-88cb-54fbe644b5fb None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:51:28.854632 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-e4c528e8-4339-4e04-88cb-54fbe644b5fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:51:28.862867 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1297/2614] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:51:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:51:28.867745 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bd98ad-f9b3-4817-bcf1-3c42f86862ec None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:28.877741 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bd98ad-f9b3-4817-bcf1-3c42f86862ec None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:28.882997 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83250de-526c-4f7d-af22-087828fa2222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:28.883664 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83250de-526c-4f7d-af22-087828fa2222 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:28.883951 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83250de-526c-4f7d-af22-087828fa2222 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:28.884227 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83250de-526c-4f7d-af22-087828fa2222 admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:28.885095 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83250de-526c-4f7d-af22-087828fa2222 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:51:28.885407 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83250de-526c-4f7d-af22-087828fa2222 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:28.887346 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83250de-526c-4f7d-af22-087828fa2222 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:28.899951 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1318/2615] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jun 16 17:51:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:51:28.904169 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c528e8-4339-4e04-88cb-54fbe644b5fb None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:28.913613 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c528e8-4339-4e04-88cb-54fbe644b5fb None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:28.918299 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4b1601-9a07-482c-b727-2e3642218cce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:28.918950 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4b1601-9a07-482c-b727-2e3642218cce admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:28.919236 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4b1601-9a07-482c-b727-2e3642218cce admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:28.919515 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4b1601-9a07-482c-b727-2e3642218cce admin admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:28.920210 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4b1601-9a07-482c-b727-2e3642218cce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:28.922166 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4b1601-9a07-482c-b727-2e3642218cce admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:28.948742 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1298/2616] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jun 16 17:51:28 2022] POST /identity/v3/projects => generated 357 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:51:28.953405 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83250de-526c-4f7d-af22-087828fa2222 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:28.964087 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83250de-526c-4f7d-af22-087828fa2222 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:28.976697 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:51:29.029177 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c79670-452a-496a-829f-fdb064539eca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:29.030028 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c79670-452a-496a-829f-fdb064539eca admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:29.030407 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c79670-452a-496a-829f-fdb064539eca admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:29.030823 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c79670-452a-496a-829f-fdb064539eca admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:29.031755 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c79670-452a-496a-829f-fdb064539eca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:29.033793 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c79670-452a-496a-829f-fdb064539eca admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:29.058619 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1319/2617] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jun 16 17:51:28 2022] POST /identity/v3/users => generated 279 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:51:29.063322 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4b1601-9a07-482c-b727-2e3642218cce admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:29.076201 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4b1601-9a07-482c-b727-2e3642218cce admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:29.088015 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f427ccd6-54b6-4cf8-8045-91c80a1ecc40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:29.088015 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f427ccd6-54b6-4cf8-8045-91c80a1ecc40 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:29.088015 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f427ccd6-54b6-4cf8-8045-91c80a1ecc40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:29.088015 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f427ccd6-54b6-4cf8-8045-91c80a1ecc40 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:29.088617 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f427ccd6-54b6-4cf8-8045-91c80a1ecc40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:29.089202 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f427ccd6-54b6-4cf8-8045-91c80a1ecc40 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:29.100909 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1299/2618] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:51:29 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:51:29.105449 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c79670-452a-496a-829f-fdb064539eca admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:29.119252 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c79670-452a-496a-829f-fdb064539eca admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:29.125485 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cc852a-9d50-47e5-a03f-3973fdbbe27b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:29.126168 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cc852a-9d50-47e5-a03f-3973fdbbe27b admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:29.126463 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cc852a-9d50-47e5-a03f-3973fdbbe27b admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:29.126730 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cc852a-9d50-47e5-a03f-3973fdbbe27b admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:29.127457 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cc852a-9d50-47e5-a03f-3973fdbbe27b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:29.129846 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cc852a-9d50-47e5-a03f-3973fdbbe27b admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:29.144810 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1320/2619] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:51:29 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:51:29.146956 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f427ccd6-54b6-4cf8-8045-91c80a1ecc40 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:29.159368 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f427ccd6-54b6-4cf8-8045-91c80a1ecc40 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:29.167172 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048bba6a-4e91-4c9f-9ab9-7cbdb13b997f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:29.168237 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048bba6a-4e91-4c9f-9ab9-7cbdb13b997f admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:29.168763 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048bba6a-4e91-4c9f-9ab9-7cbdb13b997f admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:29.169087 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048bba6a-4e91-4c9f-9ab9-7cbdb13b997f admin admin] PATH_INFO: `/v3/projects/7bed4414c8bc414bb8799ca50248a415/users/f5909897eb9b435882ff934cc1cd9742/roles/0be2075946f54c8485f1c275f746478a` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:29.188573 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048bba6a-4e91-4c9f-9ab9-7cbdb13b997f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7bed4414c8bc414bb8799ca50248a415, user_id=f5909897eb9b435882ff934cc1cd9742, role_id=0be2075946f54c8485f1c275f746478a)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:29.193378 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048bba6a-4e91-4c9f-9ab9-7cbdb13b997f admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:29.197688 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:51:29.224101 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1300/2620] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:51:29 2022] PUT /identity/v3/projects/7bed4414c8bc414bb8799ca50248a415/users/f5909897eb9b435882ff934cc1cd9742/roles/0be2075946f54c8485f1c275f746478a => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:51:29.236964 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90565fa3-6403-4b4b-9573-48df1aea38f9 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:29.237566 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90565fa3-6403-4b4b-9573-48df1aea38f9 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:29.238903 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90565fa3-6403-4b4b-9573-48df1aea38f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:29.272890 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-90565fa3-6403-4b4b-9573-48df1aea38f9 None None] MFA Rules not processed for user `f5909897eb9b435882ff934cc1cd9742`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:51:29.274122 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-90565fa3-6403-4b4b-9573-48df1aea38f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:51:29.350717 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1321/2621] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:51:29 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:51:29.355489 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048bba6a-4e91-4c9f-9ab9-7cbdb13b997f admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:29.468005 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048bba6a-4e91-4c9f-9ab9-7cbdb13b997f admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:29.475807 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ac9883-c5db-459c-abd9-1244df75a61b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:29.477436 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ac9883-c5db-459c-abd9-1244df75a61b admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:29.478126 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ac9883-c5db-459c-abd9-1244df75a61b admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:29.478987 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ac9883-c5db-459c-abd9-1244df75a61b admin admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:29.479705 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ac9883-c5db-459c-abd9-1244df75a61b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:29.481961 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ac9883-c5db-459c-abd9-1244df75a61b admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:29.514881 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1301/2622] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jun 16 17:51:29 2022] POST /identity/v3/projects => generated 357 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:51:29.519508 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90565fa3-6403-4b4b-9573-48df1aea38f9 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:29.532397 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90565fa3-6403-4b4b-9573-48df1aea38f9 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:29.610431 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767c87c9-d309-4bdd-ab36-77308e37008a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:29.611120 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767c87c9-d309-4bdd-ab36-77308e37008a admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:29.611438 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767c87c9-d309-4bdd-ab36-77308e37008a admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:29.611679 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767c87c9-d309-4bdd-ab36-77308e37008a admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:29.612391 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767c87c9-d309-4bdd-ab36-77308e37008a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:29.614404 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767c87c9-d309-4bdd-ab36-77308e37008a admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:29.639885 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1322/2623] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jun 16 17:51:29 2022] POST /identity/v3/users => generated 285 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:51:29.641252 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ac9883-c5db-459c-abd9-1244df75a61b admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:29.652654 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ac9883-c5db-459c-abd9-1244df75a61b admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:29.657506 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd29159-dcba-4ba1-8469-f2433ccf1b54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:29.658256 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd29159-dcba-4ba1-8469-f2433ccf1b54 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:29.658606 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd29159-dcba-4ba1-8469-f2433ccf1b54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:29.658945 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd29159-dcba-4ba1-8469-f2433ccf1b54 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:29.659702 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd29159-dcba-4ba1-8469-f2433ccf1b54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:29.661431 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd29159-dcba-4ba1-8469-f2433ccf1b54 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:29.672956 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1302/2624] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:51:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:51:29.677179 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767c87c9-d309-4bdd-ab36-77308e37008a admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:29.687601 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767c87c9-d309-4bdd-ab36-77308e37008a admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:29.693112 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b375a3-a964-453a-9177-1959d638131f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:29.694777 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b375a3-a964-453a-9177-1959d638131f admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:29.695061 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b375a3-a964-453a-9177-1959d638131f admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:29.695341 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b375a3-a964-453a-9177-1959d638131f admin admin] PATH_INFO: `/v3/projects/2e273d570dd044dcacd7b1293025ed46/users/a4cf9e0d150648ad98cff8fc5125e58a/roles/3ca2dcbbd29d4a31a3ab78f2299bc239` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:29.708045 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:51:29.713444 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b375a3-a964-453a-9177-1959d638131f admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e273d570dd044dcacd7b1293025ed46, user_id=a4cf9e0d150648ad98cff8fc5125e58a, role_id=3ca2dcbbd29d4a31a3ab78f2299bc239)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:29.716438 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b375a3-a964-453a-9177-1959d638131f admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:29.731187 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1323/2625] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:51:29 2022] PUT /identity/v3/projects/2e273d570dd044dcacd7b1293025ed46/users/a4cf9e0d150648ad98cff8fc5125e58a/roles/3ca2dcbbd29d4a31a3ab78f2299bc239 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:51:29.735073 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd29159-dcba-4ba1-8469-f2433ccf1b54 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:29.746404 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd29159-dcba-4ba1-8469-f2433ccf1b54 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:29.813631 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b397797-f574-4813-8083-876cb339e1c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:29.814303 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b397797-f574-4813-8083-876cb339e1c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:29.814588 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b397797-f574-4813-8083-876cb339e1c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:29.814867 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b397797-f574-4813-8083-876cb339e1c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:29.815588 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b397797-f574-4813-8083-876cb339e1c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:29.818313 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b397797-f574-4813-8083-876cb339e1c9 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:29.829932 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1303/2626] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:51:29 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:51:29.834925 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b375a3-a964-453a-9177-1959d638131f admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:29.845962 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b375a3-a964-453a-9177-1959d638131f admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:29.850868 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c9b22f-ff11-42b1-a122-9d2cd069b912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:29.851782 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c9b22f-ff11-42b1-a122-9d2cd069b912 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:29.852064 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c9b22f-ff11-42b1-a122-9d2cd069b912 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:29.852365 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c9b22f-ff11-42b1-a122-9d2cd069b912 admin admin] PATH_INFO: `/v3/projects/2e273d570dd044dcacd7b1293025ed46/users/a4cf9e0d150648ad98cff8fc5125e58a/roles/3ca2dcbbd29d4a31a3ab78f2299bc239` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:29.854626 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c9b22f-ff11-42b1-a122-9d2cd069b912 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e273d570dd044dcacd7b1293025ed46, user_id=a4cf9e0d150648ad98cff8fc5125e58a, role_id=3ca2dcbbd29d4a31a3ab78f2299bc239)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:29.857306 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c9b22f-ff11-42b1-a122-9d2cd069b912 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:29.874810 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1324/2627] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:51:29 2022] PUT /identity/v3/projects/2e273d570dd044dcacd7b1293025ed46/users/a4cf9e0d150648ad98cff8fc5125e58a/roles/3ca2dcbbd29d4a31a3ab78f2299bc239 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:51:29.889074 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11be80c9-80a8-4581-8d93-3060a8dc4d72 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:29.889373 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11be80c9-80a8-4581-8d93-3060a8dc4d72 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:29.889656 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11be80c9-80a8-4581-8d93-3060a8dc4d72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:29.923185 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-11be80c9-80a8-4581-8d93-3060a8dc4d72 None None] MFA Rules not processed for user `a4cf9e0d150648ad98cff8fc5125e58a`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:51:29.924397 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-11be80c9-80a8-4581-8d93-3060a8dc4d72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:51:29.978179 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:51:29.997941 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1304/2628] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:51:29 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:51:30.009437 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c9b22f-ff11-42b1-a122-9d2cd069b912 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:30.011173 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-f8c9b22f-ff11-42b1-a122-9d2cd069b912 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:51:30.071829 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c9b22f-ff11-42b1-a122-9d2cd069b912 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:30.076059 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0d9150-f2fe-4d89-b623-c75dfde118e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24d5e2637b7043e08313909d5b06e329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:30.076741 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0d9150-f2fe-4d89-b623-c75dfde118e2 service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:30.077029 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0d9150-f2fe-4d89-b623-c75dfde118e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:30.077310 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0d9150-f2fe-4d89-b623-c75dfde118e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:30.090577 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0d9150-f2fe-4d89-b623-c75dfde118e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:30.092083 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0d9150-f2fe-4d89-b623-c75dfde118e2 service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:30.103595 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1325/2629] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:51:30 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:51:30.175665 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11be80c9-80a8-4581-8d93-3060a8dc4d72 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:30.186593 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11be80c9-80a8-4581-8d93-3060a8dc4d72 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:30.190634 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfdf4d7-5668-44fc-8e96-e157b5ed2c7d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24d5e2637b7043e08313909d5b06e329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:30.191475 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfdf4d7-5668-44fc-8e96-e157b5ed2c7d service cinder] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:30.191755 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfdf4d7-5668-44fc-8e96-e157b5ed2c7d service cinder] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:30.192055 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfdf4d7-5668-44fc-8e96-e157b5ed2c7d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:30.252487 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfdf4d7-5668-44fc-8e96-e157b5ed2c7d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:30.254092 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfdf4d7-5668-44fc-8e96-e157b5ed2c7d service cinder] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:30.263687 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1305/2630] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:51:30 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:51:30.386480 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0d9150-f2fe-4d89-b623-c75dfde118e2 service cinder] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:30.388734 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-ea0d9150-f2fe-4d89-b623-c75dfde118e2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:51:30.462647 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0d9150-f2fe-4d89-b623-c75dfde118e2 service cinder] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:30.466941 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53a30eb-d911-4e1e-828f-8fcee119704e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:30.467598 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53a30eb-d911-4e1e-828f-8fcee119704e service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:30.467887 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53a30eb-d911-4e1e-828f-8fcee119704e service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:30.468167 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53a30eb-d911-4e1e-828f-8fcee119704e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:30.469996 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-e53a30eb-d911-4e1e-828f-8fcee119704e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:51:30.543606 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53a30eb-d911-4e1e-828f-8fcee119704e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:30.544993 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53a30eb-d911-4e1e-828f-8fcee119704e service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:30.555343 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1326/2631] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:51:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 17:51:32.529341 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a441d7b-8b7f-4c35-b54a-2d27452ac789 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:32.529872 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a441d7b-8b7f-4c35-b54a-2d27452ac789 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:32.530014 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a441d7b-8b7f-4c35-b54a-2d27452ac789 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:32.551472 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-4a441d7b-8b7f-4c35-b54a-2d27452ac789 None None] MFA Rules not processed for user `c4bffefe8fb2475a96ea033d07c790f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:51:32.553417 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-4a441d7b-8b7f-4c35-b54a-2d27452ac789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:51:32.637480 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1306/2632] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:51:32 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:51:34.617075 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53a30eb-d911-4e1e-828f-8fcee119704e service neutron] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:34.619009 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-e53a30eb-d911-4e1e-828f-8fcee119704e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:51:34.633206 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53a30eb-d911-4e1e-828f-8fcee119704e service neutron] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:34.638588 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad657fa-41d4-46d0-8885-4ce6072194fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:34.639365 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad657fa-41d4-46d0-8885-4ce6072194fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:34.639597 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad657fa-41d4-46d0-8885-4ce6072194fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:34.639887 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad657fa-41d4-46d0-8885-4ce6072194fc admin admin] PATH_INFO: `/v3/users/7b078c6bfca04de790e7b2e45739033e` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:34.641254 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad657fa-41d4-46d0-8885-4ce6072194fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b078c6bfca04de790e7b2e45739033e)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:34.643174 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad657fa-41d4-46d0-8885-4ce6072194fc admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:34.671236 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:51:34.714128 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-6ad657fa-41d4-46d0-8885-4ce6072194fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b078c6bfca04de790e7b2e45739033e'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:34.744747 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-6ad657fa-41d4-46d0-8885-4ce6072194fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b078c6bfca04de790e7b2e45739033e'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:34.745698 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-6ad657fa-41d4-46d0-8885-4ce6072194fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b078c6bfca04de790e7b2e45739033e'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:34.755624 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-6ad657fa-41d4-46d0-8885-4ce6072194fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b078c6bfca04de790e7b2e45739033e'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:34.770661 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-6ad657fa-41d4-46d0-8885-4ce6072194fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b078c6bfca04de790e7b2e45739033e'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:34.780422 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1327/2633] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jun 16 17:51:34 2022] DELETE /identity/v3/users/7b078c6bfca04de790e7b2e45739033e => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:51:35.048104 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a441d7b-8b7f-4c35-b54a-2d27452ac789 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:35.064688 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a441d7b-8b7f-4c35-b54a-2d27452ac789 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:35.129306 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a2a8f6-6c69-439b-a11b-f62aa59ea1f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:35.130281 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a2a8f6-6c69-439b-a11b-f62aa59ea1f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:35.130681 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a2a8f6-6c69-439b-a11b-f62aa59ea1f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:35.130851 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a2a8f6-6c69-439b-a11b-f62aa59ea1f2 admin admin] PATH_INFO: `/v3/projects/bc6671e3a8bd4821b17ef2d2a426ee58` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:35.132840 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a2a8f6-6c69-439b-a11b-f62aa59ea1f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc6671e3a8bd4821b17ef2d2a426ee58)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:35.135236 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a2a8f6-6c69-439b-a11b-f62aa59ea1f2 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:35.181923 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:51:35.192411 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-76a2a8f6-6c69-439b-a11b-f62aa59ea1f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc6671e3a8bd4821b17ef2d2a426ee58', 'request_id': 'req-76a2a8f6-6c69-439b-a11b-f62aa59ea1f2'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:35.201333 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-76a2a8f6-6c69-439b-a11b-f62aa59ea1f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc6671e3a8bd4821b17ef2d2a426ee58', 'request_id': 'req-76a2a8f6-6c69-439b-a11b-f62aa59ea1f2'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:35.208493 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-76a2a8f6-6c69-439b-a11b-f62aa59ea1f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc6671e3a8bd4821b17ef2d2a426ee58', 'request_id': 'req-76a2a8f6-6c69-439b-a11b-f62aa59ea1f2'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:35.220503 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-76a2a8f6-6c69-439b-a11b-f62aa59ea1f2 admin admin] The token cache is being invalidate because project bc6671e3a8bd4821b17ef2d2a426ee58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80357) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:51:35.220796 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-76a2a8f6-6c69-439b-a11b-f62aa59ea1f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:35.221499 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-76a2a8f6-6c69-439b-a11b-f62aa59ea1f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:35.223914 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1307/2634] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jun 16 17:51:35 2022] DELETE /identity/v3/projects/bc6671e3a8bd4821b17ef2d2a426ee58 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:51:35.228896 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be49a5d-1471-47d2-905d-a435699ac9e0 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:35.229302 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be49a5d-1471-47d2-905d-a435699ac9e0 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:35.229696 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be49a5d-1471-47d2-905d-a435699ac9e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:35.251107 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-6be49a5d-1471-47d2-905d-a435699ac9e0 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:51:35.252460 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-6be49a5d-1471-47d2-905d-a435699ac9e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:51:35.323258 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1328/2635] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:51:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:51:35.335535 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091d91fe-4fde-4b28-823d-f25d759632ca None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:35.335979 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091d91fe-4fde-4b28-823d-f25d759632ca None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:35.336441 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091d91fe-4fde-4b28-823d-f25d759632ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:35.366231 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-091d91fe-4fde-4b28-823d-f25d759632ca None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:51:35.368109 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-091d91fe-4fde-4b28-823d-f25d759632ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:51:35.385228 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1308/2636] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:51:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:51:35.390514 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be49a5d-1471-47d2-905d-a435699ac9e0 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:35.403899 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be49a5d-1471-47d2-905d-a435699ac9e0 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:35.408833 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e732c2-6a10-4c9d-b5b3-21dfde654d30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:35.410124 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e732c2-6a10-4c9d-b5b3-21dfde654d30 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:35.410793 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e732c2-6a10-4c9d-b5b3-21dfde654d30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:35.410793 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e732c2-6a10-4c9d-b5b3-21dfde654d30 admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:35.411402 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e732c2-6a10-4c9d-b5b3-21dfde654d30 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:51:35.411715 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e732c2-6a10-4c9d-b5b3-21dfde654d30 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:35.415556 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e732c2-6a10-4c9d-b5b3-21dfde654d30 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:35.430074 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1329/2637] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jun 16 17:51:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:51:35.431250 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091d91fe-4fde-4b28-823d-f25d759632ca None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:35.443172 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091d91fe-4fde-4b28-823d-f25d759632ca None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:35.448300 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b710843f-ab2a-4a1b-bff9-ceb46e29bafd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:35.449094 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b710843f-ab2a-4a1b-bff9-ceb46e29bafd admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:35.449738 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b710843f-ab2a-4a1b-bff9-ceb46e29bafd admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:35.450062 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b710843f-ab2a-4a1b-bff9-ceb46e29bafd admin admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:35.450743 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b710843f-ab2a-4a1b-bff9-ceb46e29bafd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:35.453052 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b710843f-ab2a-4a1b-bff9-ceb46e29bafd admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:35.491693 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1309/2638] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jun 16 17:51:35 2022] POST /identity/v3/projects => generated 383 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:51:35.493165 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e732c2-6a10-4c9d-b5b3-21dfde654d30 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:35.504734 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e732c2-6a10-4c9d-b5b3-21dfde654d30 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:35.569639 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91c616f-e8e9-4058-a03c-f034f10ed0b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:35.570446 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91c616f-e8e9-4058-a03c-f034f10ed0b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:35.570809 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91c616f-e8e9-4058-a03c-f034f10ed0b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:35.571204 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91c616f-e8e9-4058-a03c-f034f10ed0b7 admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:35.571954 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91c616f-e8e9-4058-a03c-f034f10ed0b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:35.574152 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91c616f-e8e9-4058-a03c-f034f10ed0b7 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:35.597927 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1330/2639] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jun 16 17:51:35 2022] POST /identity/v3/users => generated 292 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:51:35.603856 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b710843f-ab2a-4a1b-bff9-ceb46e29bafd admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:35.615080 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b710843f-ab2a-4a1b-bff9-ceb46e29bafd admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:35.620915 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77ee81f-7c46-4b07-be84-847d35ea05ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:35.621723 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77ee81f-7c46-4b07-be84-847d35ea05ef admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:35.621988 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77ee81f-7c46-4b07-be84-847d35ea05ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:35.622300 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77ee81f-7c46-4b07-be84-847d35ea05ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:35.623039 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77ee81f-7c46-4b07-be84-847d35ea05ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:35.624716 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77ee81f-7c46-4b07-be84-847d35ea05ef admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:35.637649 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1310/2640] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:51:35 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 17:51:35.641760 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91c616f-e8e9-4058-a03c-f034f10ed0b7 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:35.652919 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91c616f-e8e9-4058-a03c-f034f10ed0b7 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:35.659305 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5ec391-ac89-4d12-ab7d-b756a48bf7ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:35.660870 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5ec391-ac89-4d12-ab7d-b756a48bf7ad admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:35.661251 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5ec391-ac89-4d12-ab7d-b756a48bf7ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:35.661597 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5ec391-ac89-4d12-ab7d-b756a48bf7ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:35.662408 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5ec391-ac89-4d12-ab7d-b756a48bf7ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:35.664286 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5ec391-ac89-4d12-ab7d-b756a48bf7ad admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:35.676362 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1331/2641] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:51:35 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:51:35.680693 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77ee81f-7c46-4b07-be84-847d35ea05ef admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:35.692173 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77ee81f-7c46-4b07-be84-847d35ea05ef admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:35.696959 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc863371-48e2-4ce0-86f8-968c4854a3f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:35.697780 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc863371-48e2-4ce0-86f8-968c4854a3f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:35.698064 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc863371-48e2-4ce0-86f8-968c4854a3f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:35.698345 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc863371-48e2-4ce0-86f8-968c4854a3f1 admin admin] PATH_INFO: `/v3/projects/6c2eb3d7f65a4f80999395271ed2c850/users/a2f25294bba44cf8a5aaeafce5f63815/roles/0be2075946f54c8485f1c275f746478a` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:35.717777 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc863371-48e2-4ce0-86f8-968c4854a3f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c2eb3d7f65a4f80999395271ed2c850, user_id=a2f25294bba44cf8a5aaeafce5f63815, role_id=0be2075946f54c8485f1c275f746478a)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:35.720583 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc863371-48e2-4ce0-86f8-968c4854a3f1 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:35.740547 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1311/2642] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:51:35 2022] PUT /identity/v3/projects/6c2eb3d7f65a4f80999395271ed2c850/users/a2f25294bba44cf8a5aaeafce5f63815/roles/0be2075946f54c8485f1c275f746478a => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:51:35.751901 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a21e4c-77d0-4878-8aae-9f0c52b44acc None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:35.751901 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a21e4c-77d0-4878-8aae-9f0c52b44acc None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:35.751901 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a21e4c-77d0-4878-8aae-9f0c52b44acc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:35.770496 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:51:35.793942 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-91a21e4c-77d0-4878-8aae-9f0c52b44acc None None] MFA Rules not processed for user `a2f25294bba44cf8a5aaeafce5f63815`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:51:35.795174 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-91a21e4c-77d0-4878-8aae-9f0c52b44acc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:51:35.877863 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1332/2643] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:51:35 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:51:35.891101 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc863371-48e2-4ce0-86f8-968c4854a3f1 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:35.892999 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-bc863371-48e2-4ce0-86f8-968c4854a3f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:51:35.991127 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc863371-48e2-4ce0-86f8-968c4854a3f1 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:35.997581 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c19ae29-4441-48d3-ad9a-a2a92c524159 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f86c16179a70474a817be9fda33ece9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:35.998410 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c19ae29-4441-48d3-ad9a-a2a92c524159 service nova] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:35.998880 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c19ae29-4441-48d3-ad9a-a2a92c524159 service nova] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:35.999162 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c19ae29-4441-48d3-ad9a-a2a92c524159 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:36.009284 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c19ae29-4441-48d3-ad9a-a2a92c524159 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:36.011094 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c19ae29-4441-48d3-ad9a-a2a92c524159 service nova] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:36.023591 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1312/2644] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:51:35 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:51:44.481548 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a21e4c-77d0-4878-8aae-9f0c52b44acc None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:44.546962 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a21e4c-77d0-4878-8aae-9f0c52b44acc None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:44.552309 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910cc586-beba-4cea-8f9e-3d3f850c8ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:44.553229 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910cc586-beba-4cea-8f9e-3d3f850c8ee3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:44.553512 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910cc586-beba-4cea-8f9e-3d3f850c8ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:44.553813 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910cc586-beba-4cea-8f9e-3d3f850c8ee3 admin admin] PATH_INFO: `/v3/users/a2f25294bba44cf8a5aaeafce5f63815` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:44.555040 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910cc586-beba-4cea-8f9e-3d3f850c8ee3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2f25294bba44cf8a5aaeafce5f63815)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:44.557107 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910cc586-beba-4cea-8f9e-3d3f850c8ee3 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:44.603706 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-910cc586-beba-4cea-8f9e-3d3f850c8ee3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2f25294bba44cf8a5aaeafce5f63815'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:44.616854 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-910cc586-beba-4cea-8f9e-3d3f850c8ee3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2f25294bba44cf8a5aaeafce5f63815'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:44.617594 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-910cc586-beba-4cea-8f9e-3d3f850c8ee3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2f25294bba44cf8a5aaeafce5f63815'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:44.624998 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-910cc586-beba-4cea-8f9e-3d3f850c8ee3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2f25294bba44cf8a5aaeafce5f63815'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:44.634804 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-910cc586-beba-4cea-8f9e-3d3f850c8ee3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2f25294bba44cf8a5aaeafce5f63815'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:44.641489 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1333/2645] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jun 16 17:51:44 2022] DELETE /identity/v3/users/a2f25294bba44cf8a5aaeafce5f63815 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:51:44.646131 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53abb574-1de1-4e1a-9d08-691066bdab5e None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:44.646642 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53abb574-1de1-4e1a-9d08-691066bdab5e None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:44.646762 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53abb574-1de1-4e1a-9d08-691066bdab5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:44.664181 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-53abb574-1de1-4e1a-9d08-691066bdab5e None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:51:44.665420 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-53abb574-1de1-4e1a-9d08-691066bdab5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:51:44.710052 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:51:44.730498 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1313/2646] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:51:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:51:44.740750 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910cc586-beba-4cea-8f9e-3d3f850c8ee3 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:44.742246 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-910cc586-beba-4cea-8f9e-3d3f850c8ee3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:51:44.762838 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:51:44.806523 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910cc586-beba-4cea-8f9e-3d3f850c8ee3 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:44.810321 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a8fb3b-55fc-4ae9-9508-4e0fe0ac42b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:44.810976 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a8fb3b-55fc-4ae9-9508-4e0fe0ac42b1 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:44.811269 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a8fb3b-55fc-4ae9-9508-4e0fe0ac42b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:44.811548 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a8fb3b-55fc-4ae9-9508-4e0fe0ac42b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:44.822511 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a8fb3b-55fc-4ae9-9508-4e0fe0ac42b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:44.824016 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a8fb3b-55fc-4ae9-9508-4e0fe0ac42b1 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:44.835376 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1334/2647] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:51:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 17:51:45.138179 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53abb574-1de1-4e1a-9d08-691066bdab5e None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:45.139852 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-53abb574-1de1-4e1a-9d08-691066bdab5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:51:45.153522 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53abb574-1de1-4e1a-9d08-691066bdab5e None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:45.159168 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2920b5df-f671-44f0-9411-a79090c59d72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:45.160633 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2920b5df-f671-44f0-9411-a79090c59d72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:45.162066 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2920b5df-f671-44f0-9411-a79090c59d72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:45.162735 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2920b5df-f671-44f0-9411-a79090c59d72 admin admin] PATH_INFO: `/v3/projects/6c2eb3d7f65a4f80999395271ed2c850` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:45.164041 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2920b5df-f671-44f0-9411-a79090c59d72 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c2eb3d7f65a4f80999395271ed2c850)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:45.166087 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2920b5df-f671-44f0-9411-a79090c59d72 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:45.224411 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-2920b5df-f671-44f0-9411-a79090c59d72 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c2eb3d7f65a4f80999395271ed2c850', 'request_id': 'req-2920b5df-f671-44f0-9411-a79090c59d72'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:45.233986 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-2920b5df-f671-44f0-9411-a79090c59d72 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c2eb3d7f65a4f80999395271ed2c850', 'request_id': 'req-2920b5df-f671-44f0-9411-a79090c59d72'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:45.240750 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-2920b5df-f671-44f0-9411-a79090c59d72 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c2eb3d7f65a4f80999395271ed2c850', 'request_id': 'req-2920b5df-f671-44f0-9411-a79090c59d72'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:45.251777 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-2920b5df-f671-44f0-9411-a79090c59d72 admin admin] The token cache is being invalidate because project 6c2eb3d7f65a4f80999395271ed2c850 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80357) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:51:45.252066 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-2920b5df-f671-44f0-9411-a79090c59d72 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:45.252754 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-2920b5df-f671-44f0-9411-a79090c59d72 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:45.255350 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1314/2648] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jun 16 17:51:45 2022] DELETE /identity/v3/projects/6c2eb3d7f65a4f80999395271ed2c850 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:51:45.260235 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17ba0c9-ebaa-4b26-aa92-26241c35bbf2 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:45.260580 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17ba0c9-ebaa-4b26-aa92-26241c35bbf2 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:45.260866 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17ba0c9-ebaa-4b26-aa92-26241c35bbf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:45.282414 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-c17ba0c9-ebaa-4b26-aa92-26241c35bbf2 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:51:45.283681 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-c17ba0c9-ebaa-4b26-aa92-26241c35bbf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:51:45.355434 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1335/2649] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:51:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:51:45.362881 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9370f9a9-7d8f-4900-8cb2-05b56a6ec245 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:45.363200 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9370f9a9-7d8f-4900-8cb2-05b56a6ec245 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:45.363493 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9370f9a9-7d8f-4900-8cb2-05b56a6ec245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:45.383110 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-9370f9a9-7d8f-4900-8cb2-05b56a6ec245 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:51:45.384321 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-9370f9a9-7d8f-4900-8cb2-05b56a6ec245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:51:45.392486 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1315/2650] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:51:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:51:45.397787 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17ba0c9-ebaa-4b26-aa92-26241c35bbf2 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:45.407613 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17ba0c9-ebaa-4b26-aa92-26241c35bbf2 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:45.413596 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2249917b-e4b3-400d-a4fe-12da468d1d6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:45.414247 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2249917b-e4b3-400d-a4fe-12da468d1d6b admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:45.414564 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2249917b-e4b3-400d-a4fe-12da468d1d6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:45.414851 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2249917b-e4b3-400d-a4fe-12da468d1d6b admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:45.416221 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2249917b-e4b3-400d-a4fe-12da468d1d6b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:51:45.416578 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2249917b-e4b3-400d-a4fe-12da468d1d6b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:45.418216 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2249917b-e4b3-400d-a4fe-12da468d1d6b admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:45.432114 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1336/2651] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jun 16 17:51:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:51:45.437899 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9370f9a9-7d8f-4900-8cb2-05b56a6ec245 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:45.448958 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9370f9a9-7d8f-4900-8cb2-05b56a6ec245 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:45.453707 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7247c22d-4e71-43f2-900f-a716d1a82230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:45.454415 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7247c22d-4e71-43f2-900f-a716d1a82230 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:45.454695 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7247c22d-4e71-43f2-900f-a716d1a82230 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:45.454970 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7247c22d-4e71-43f2-900f-a716d1a82230 admin admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:45.455696 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7247c22d-4e71-43f2-900f-a716d1a82230 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:45.457896 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7247c22d-4e71-43f2-900f-a716d1a82230 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:45.492022 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1316/2652] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jun 16 17:51:45 2022] POST /identity/v3/projects => generated 365 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:51:45.493112 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2249917b-e4b3-400d-a4fe-12da468d1d6b admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:45.503071 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2249917b-e4b3-400d-a4fe-12da468d1d6b admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:45.572733 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fa1ee6-ff7a-4aec-b602-197f89934b9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:45.573424 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fa1ee6-ff7a-4aec-b602-197f89934b9d admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:45.573709 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fa1ee6-ff7a-4aec-b602-197f89934b9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:45.573987 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fa1ee6-ff7a-4aec-b602-197f89934b9d admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:45.574694 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fa1ee6-ff7a-4aec-b602-197f89934b9d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:45.576676 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fa1ee6-ff7a-4aec-b602-197f89934b9d admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:45.600057 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1337/2653] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jun 16 17:51:45 2022] POST /identity/v3/users => generated 283 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:51:45.604193 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7247c22d-4e71-43f2-900f-a716d1a82230 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:45.619602 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7247c22d-4e71-43f2-900f-a716d1a82230 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:45.624738 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89247344-7362-45ab-8241-a7d1bb01fb53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:45.625409 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89247344-7362-45ab-8241-a7d1bb01fb53 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:45.625694 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89247344-7362-45ab-8241-a7d1bb01fb53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:45.625973 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89247344-7362-45ab-8241-a7d1bb01fb53 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:45.626709 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89247344-7362-45ab-8241-a7d1bb01fb53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:45.628337 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89247344-7362-45ab-8241-a7d1bb01fb53 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:45.641283 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1317/2654] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:51:45 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:51:45.646535 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fa1ee6-ff7a-4aec-b602-197f89934b9d admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:45.656993 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fa1ee6-ff7a-4aec-b602-197f89934b9d admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:45.662979 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bd1ac5-6249-4113-b8bb-b4e8637807b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:45.663739 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bd1ac5-6249-4113-b8bb-b4e8637807b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:45.664029 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bd1ac5-6249-4113-b8bb-b4e8637807b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:45.664313 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bd1ac5-6249-4113-b8bb-b4e8637807b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:45.665087 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bd1ac5-6249-4113-b8bb-b4e8637807b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:45.666690 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bd1ac5-6249-4113-b8bb-b4e8637807b5 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:45.678144 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1338/2655] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:51:45 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 17:51:45.682670 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89247344-7362-45ab-8241-a7d1bb01fb53 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:45.693331 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89247344-7362-45ab-8241-a7d1bb01fb53 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:45.698903 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad91d6e-378a-4926-83fe-17d9a169b4bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:45.699799 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad91d6e-378a-4926-83fe-17d9a169b4bc admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:45.700087 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad91d6e-378a-4926-83fe-17d9a169b4bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:45.700397 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad91d6e-378a-4926-83fe-17d9a169b4bc admin admin] PATH_INFO: `/v3/projects/e6a2a4d7477841f0879ab8c2be04f7b2/users/0723640a0162478dbff3f194f4923362/roles/0be2075946f54c8485f1c275f746478a` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:45.716243 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad91d6e-378a-4926-83fe-17d9a169b4bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6a2a4d7477841f0879ab8c2be04f7b2, user_id=0723640a0162478dbff3f194f4923362, role_id=0be2075946f54c8485f1c275f746478a)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:45.718938 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad91d6e-378a-4926-83fe-17d9a169b4bc admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:45.732160 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1318/2656] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:51:45 2022] PUT /identity/v3/projects/e6a2a4d7477841f0879ab8c2be04f7b2/users/0723640a0162478dbff3f194f4923362/roles/0be2075946f54c8485f1c275f746478a => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:51:45.744397 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbde2d6-f818-4109-8802-28de53337a0c None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:45.744741 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbde2d6-f818-4109-8802-28de53337a0c None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:45.745034 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbde2d6-f818-4109-8802-28de53337a0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:45.774989 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-edbde2d6-f818-4109-8802-28de53337a0c None None] MFA Rules not processed for user `0723640a0162478dbff3f194f4923362`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:51:45.776270 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-edbde2d6-f818-4109-8802-28de53337a0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:51:45.789301 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:51:45.857776 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1339/2657] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:51:45 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:51:45.863826 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad91d6e-378a-4926-83fe-17d9a169b4bc admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:45.934008 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad91d6e-378a-4926-83fe-17d9a169b4bc admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:45.941080 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763983b6-5607-43ef-a622-288b7fb903a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:45.942460 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763983b6-5607-43ef-a622-288b7fb903a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:45.946529 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763983b6-5607-43ef-a622-288b7fb903a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:45.946529 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763983b6-5607-43ef-a622-288b7fb903a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:45.946529 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763983b6-5607-43ef-a622-288b7fb903a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:45.946529 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763983b6-5607-43ef-a622-288b7fb903a7 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:45.956196 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:51:45.976169 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1319/2658] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jun 16 17:51:45 2022] POST /identity/v3/projects => generated 363 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:51:45.981745 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbde2d6-f818-4109-8802-28de53337a0c None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:45.999066 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbde2d6-f818-4109-8802-28de53337a0c None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:46.062705 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140bb126-9128-4901-ac57-2cf313702e03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:46.063398 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140bb126-9128-4901-ac57-2cf313702e03 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:46.063687 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140bb126-9128-4901-ac57-2cf313702e03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:46.064045 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140bb126-9128-4901-ac57-2cf313702e03 admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:46.064906 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140bb126-9128-4901-ac57-2cf313702e03 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:46.066963 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140bb126-9128-4901-ac57-2cf313702e03 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:46.089930 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1340/2659] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jun 16 17:51:45 2022] POST /identity/v3/users => generated 288 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:51:46.094224 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763983b6-5607-43ef-a622-288b7fb903a7 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:46.106772 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763983b6-5607-43ef-a622-288b7fb903a7 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:46.113090 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67db59fb-4597-4d84-a14e-b9fa8887cb0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:46.114756 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67db59fb-4597-4d84-a14e-b9fa8887cb0d admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:46.115114 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67db59fb-4597-4d84-a14e-b9fa8887cb0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:46.115431 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67db59fb-4597-4d84-a14e-b9fa8887cb0d admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:46.116503 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67db59fb-4597-4d84-a14e-b9fa8887cb0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:46.118652 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67db59fb-4597-4d84-a14e-b9fa8887cb0d admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:46.139033 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1320/2660] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:51:46 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:51:46.141765 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140bb126-9128-4901-ac57-2cf313702e03 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:46.154542 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140bb126-9128-4901-ac57-2cf313702e03 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:46.162174 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f67337f-38af-4dfd-82de-140750a92f7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:46.163703 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f67337f-38af-4dfd-82de-140750a92f7b admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:46.164123 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f67337f-38af-4dfd-82de-140750a92f7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:46.164760 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f67337f-38af-4dfd-82de-140750a92f7b admin admin] PATH_INFO: `/v3/projects/8f7569d32caf4229ae825a21633bbc58/users/d3599cf0162f479d8a138bdc618313c7/roles/3ca2dcbbd29d4a31a3ab78f2299bc239` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:46.195603 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f67337f-38af-4dfd-82de-140750a92f7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f7569d32caf4229ae825a21633bbc58, user_id=d3599cf0162f479d8a138bdc618313c7, role_id=3ca2dcbbd29d4a31a3ab78f2299bc239)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:46.196739 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f67337f-38af-4dfd-82de-140750a92f7b admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:46.215981 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1341/2661] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:51:46 2022] PUT /identity/v3/projects/8f7569d32caf4229ae825a21633bbc58/users/d3599cf0162f479d8a138bdc618313c7/roles/3ca2dcbbd29d4a31a3ab78f2299bc239 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:51:46.220754 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67db59fb-4597-4d84-a14e-b9fa8887cb0d admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:46.243960 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67db59fb-4597-4d84-a14e-b9fa8887cb0d admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:46.347016 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0c2312-d3b9-4563-84ef-6bea3c675928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:46.348712 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0c2312-d3b9-4563-84ef-6bea3c675928 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:46.349012 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0c2312-d3b9-4563-84ef-6bea3c675928 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:46.349443 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0c2312-d3b9-4563-84ef-6bea3c675928 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:46.351057 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0c2312-d3b9-4563-84ef-6bea3c675928 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:46.351830 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0c2312-d3b9-4563-84ef-6bea3c675928 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:46.366753 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1321/2662] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:51:46 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:51:46.371523 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f67337f-38af-4dfd-82de-140750a92f7b admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:46.392394 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f67337f-38af-4dfd-82de-140750a92f7b admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:46.409028 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1a1c28-c4c6-4098-8291-dc0bcf6df148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:46.411477 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1a1c28-c4c6-4098-8291-dc0bcf6df148 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:46.411872 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1a1c28-c4c6-4098-8291-dc0bcf6df148 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:46.412178 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1a1c28-c4c6-4098-8291-dc0bcf6df148 admin admin] PATH_INFO: `/v3/projects/8f7569d32caf4229ae825a21633bbc58/users/d3599cf0162f479d8a138bdc618313c7/roles/3ca2dcbbd29d4a31a3ab78f2299bc239` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:46.414675 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1a1c28-c4c6-4098-8291-dc0bcf6df148 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f7569d32caf4229ae825a21633bbc58, user_id=d3599cf0162f479d8a138bdc618313c7, role_id=3ca2dcbbd29d4a31a3ab78f2299bc239)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:46.417565 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1a1c28-c4c6-4098-8291-dc0bcf6df148 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:46.434694 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1342/2663] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:51:46 2022] PUT /identity/v3/projects/8f7569d32caf4229ae825a21633bbc58/users/d3599cf0162f479d8a138bdc618313c7/roles/3ca2dcbbd29d4a31a3ab78f2299bc239 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:51:46.448014 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d5bec3-0251-4cd7-9f84-a041a3c8eddc None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:46.448323 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d5bec3-0251-4cd7-9f84-a041a3c8eddc None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:46.448626 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d5bec3-0251-4cd7-9f84-a041a3c8eddc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:46.489051 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-57d5bec3-0251-4cd7-9f84-a041a3c8eddc None None] MFA Rules not processed for user `d3599cf0162f479d8a138bdc618313c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:51:46.490027 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-57d5bec3-0251-4cd7-9f84-a041a3c8eddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:51:46.602138 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1322/2664] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:51:46 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:51:46.619127 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1a1c28-c4c6-4098-8291-dc0bcf6df148 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:46.621902 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-9e1a1c28-c4c6-4098-8291-dc0bcf6df148 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:51:46.696426 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1a1c28-c4c6-4098-8291-dc0bcf6df148 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:46.701587 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865295cf-908b-41ae-96d8-3238179904ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24d5e2637b7043e08313909d5b06e329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:46.702345 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865295cf-908b-41ae-96d8-3238179904ae service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:46.702662 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865295cf-908b-41ae-96d8-3238179904ae service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:46.702938 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865295cf-908b-41ae-96d8-3238179904ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:46.714783 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865295cf-908b-41ae-96d8-3238179904ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:46.716996 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865295cf-908b-41ae-96d8-3238179904ae service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:46.728798 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1343/2665] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:51:46 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:51:46.808874 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d5bec3-0251-4cd7-9f84-a041a3c8eddc None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:46.821346 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d5bec3-0251-4cd7-9f84-a041a3c8eddc None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:46.825393 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454c2ab6-7834-4d8d-ad43-f06e5f8196b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24d5e2637b7043e08313909d5b06e329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:46.826055 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454c2ab6-7834-4d8d-ad43-f06e5f8196b3 service cinder] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:46.826341 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454c2ab6-7834-4d8d-ad43-f06e5f8196b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:46.826619 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454c2ab6-7834-4d8d-ad43-f06e5f8196b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:46.913887 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454c2ab6-7834-4d8d-ad43-f06e5f8196b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:46.916226 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454c2ab6-7834-4d8d-ad43-f06e5f8196b3 service cinder] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:46.922454 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:51:46.930673 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1323/2666] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:51:46 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:51:48.037285 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865295cf-908b-41ae-96d8-3238179904ae service cinder] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:48.038775 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-865295cf-908b-41ae-96d8-3238179904ae service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:51:48.049700 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:51:48.114804 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865295cf-908b-41ae-96d8-3238179904ae service cinder] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:48.119691 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4571463a-af2d-4f04-8f50-e3823390692b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:48.120708 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4571463a-af2d-4f04-8f50-e3823390692b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:48.121079 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4571463a-af2d-4f04-8f50-e3823390692b admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:48.121436 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4571463a-af2d-4f04-8f50-e3823390692b admin admin] PATH_INFO: `/v3/users/f5909897eb9b435882ff934cc1cd9742` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:48.122747 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4571463a-af2d-4f04-8f50-e3823390692b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5909897eb9b435882ff934cc1cd9742)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:48.124716 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4571463a-af2d-4f04-8f50-e3823390692b admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:48.185474 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-4571463a-af2d-4f04-8f50-e3823390692b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5909897eb9b435882ff934cc1cd9742'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:48.200391 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-4571463a-af2d-4f04-8f50-e3823390692b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5909897eb9b435882ff934cc1cd9742'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:48.201253 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-4571463a-af2d-4f04-8f50-e3823390692b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5909897eb9b435882ff934cc1cd9742'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:48.213912 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-4571463a-af2d-4f04-8f50-e3823390692b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5909897eb9b435882ff934cc1cd9742'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:48.224654 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-4571463a-af2d-4f04-8f50-e3823390692b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5909897eb9b435882ff934cc1cd9742'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:48.231160 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1344/2667] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jun 16 17:51:48 2022] DELETE /identity/v3/users/f5909897eb9b435882ff934cc1cd9742 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:51:48.235684 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39417f56-fbbf-44f1-8db5-054827932e54 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:48.236173 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39417f56-fbbf-44f1-8db5-054827932e54 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:48.236274 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39417f56-fbbf-44f1-8db5-054827932e54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:48.257302 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-39417f56-fbbf-44f1-8db5-054827932e54 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:51:48.258522 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-39417f56-fbbf-44f1-8db5-054827932e54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:51:48.339038 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1324/2668] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:51:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:51:48.350525 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4571463a-af2d-4f04-8f50-e3823390692b admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:48.352782 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-4571463a-af2d-4f04-8f50-e3823390692b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:51:48.424786 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4571463a-af2d-4f04-8f50-e3823390692b admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:48.430435 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89172df-0dc9-400f-8659-9d3924b7d8cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:48.431170 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89172df-0dc9-400f-8659-9d3924b7d8cf service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:48.431527 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89172df-0dc9-400f-8659-9d3924b7d8cf service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:48.431858 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89172df-0dc9-400f-8659-9d3924b7d8cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:48.448806 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89172df-0dc9-400f-8659-9d3924b7d8cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:48.450853 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89172df-0dc9-400f-8659-9d3924b7d8cf service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:48.490749 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1345/2669] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:51:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:51:48.822084 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39417f56-fbbf-44f1-8db5-054827932e54 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:48.823746 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-39417f56-fbbf-44f1-8db5-054827932e54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:51:48.870099 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39417f56-fbbf-44f1-8db5-054827932e54 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:48.876161 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2280efe6-519d-42bc-9e9b-a98e9e46f57c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:48.877254 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2280efe6-519d-42bc-9e9b-a98e9e46f57c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:48.877563 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2280efe6-519d-42bc-9e9b-a98e9e46f57c admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:48.878004 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2280efe6-519d-42bc-9e9b-a98e9e46f57c admin admin] PATH_INFO: `/v3/projects/7bed4414c8bc414bb8799ca50248a415` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:48.879417 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2280efe6-519d-42bc-9e9b-a98e9e46f57c admin admin] RBAC: Authorizing `identity:delete_project(project_id=7bed4414c8bc414bb8799ca50248a415)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:48.884426 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2280efe6-519d-42bc-9e9b-a98e9e46f57c admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:48.943643 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-2280efe6-519d-42bc-9e9b-a98e9e46f57c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7bed4414c8bc414bb8799ca50248a415', 'request_id': 'req-2280efe6-519d-42bc-9e9b-a98e9e46f57c'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:48.951003 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-2280efe6-519d-42bc-9e9b-a98e9e46f57c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7bed4414c8bc414bb8799ca50248a415', 'request_id': 'req-2280efe6-519d-42bc-9e9b-a98e9e46f57c'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:48.956471 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-2280efe6-519d-42bc-9e9b-a98e9e46f57c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7bed4414c8bc414bb8799ca50248a415', 'request_id': 'req-2280efe6-519d-42bc-9e9b-a98e9e46f57c'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:48.968138 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-2280efe6-519d-42bc-9e9b-a98e9e46f57c admin admin] The token cache is being invalidate because project 7bed4414c8bc414bb8799ca50248a415 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80357) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:51:48.968456 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-2280efe6-519d-42bc-9e9b-a98e9e46f57c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:48.969082 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-2280efe6-519d-42bc-9e9b-a98e9e46f57c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:48.970623 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1325/2670] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jun 16 17:51:48 2022] DELETE /identity/v3/projects/7bed4414c8bc414bb8799ca50248a415 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:51:48.974864 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89172df-0dc9-400f-8659-9d3924b7d8cf service neutron] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:48.984825 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89172df-0dc9-400f-8659-9d3924b7d8cf service neutron] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:49.016262 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:51:49.045134 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be2a447-ee73-4411-8df5-6700a51f830b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:49.046006 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be2a447-ee73-4411-8df5-6700a51f830b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:49.046288 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be2a447-ee73-4411-8df5-6700a51f830b admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:49.046562 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be2a447-ee73-4411-8df5-6700a51f830b admin admin] PATH_INFO: `/v3/users/a4cf9e0d150648ad98cff8fc5125e58a` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:49.047753 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be2a447-ee73-4411-8df5-6700a51f830b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4cf9e0d150648ad98cff8fc5125e58a)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:49.049693 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be2a447-ee73-4411-8df5-6700a51f830b admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:49.098296 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-3be2a447-ee73-4411-8df5-6700a51f830b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4cf9e0d150648ad98cff8fc5125e58a'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:49.110107 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-3be2a447-ee73-4411-8df5-6700a51f830b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4cf9e0d150648ad98cff8fc5125e58a'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:49.110792 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-3be2a447-ee73-4411-8df5-6700a51f830b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4cf9e0d150648ad98cff8fc5125e58a'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:49.117926 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-3be2a447-ee73-4411-8df5-6700a51f830b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4cf9e0d150648ad98cff8fc5125e58a'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:49.125263 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-3be2a447-ee73-4411-8df5-6700a51f830b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4cf9e0d150648ad98cff8fc5125e58a'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:49.132059 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1346/2671] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jun 16 17:51:48 2022] DELETE /identity/v3/users/a4cf9e0d150648ad98cff8fc5125e58a => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:51:49.421572 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2280efe6-519d-42bc-9e9b-a98e9e46f57c admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:49.423141 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-2280efe6-519d-42bc-9e9b-a98e9e46f57c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:51:49.439044 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2280efe6-519d-42bc-9e9b-a98e9e46f57c admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:49.506943 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fc0bc5-e2aa-4532-bcaf-00ec9d63ab2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:49.507806 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fc0bc5-e2aa-4532-bcaf-00ec9d63ab2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:49.508089 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fc0bc5-e2aa-4532-bcaf-00ec9d63ab2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:49.508395 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fc0bc5-e2aa-4532-bcaf-00ec9d63ab2f admin admin] PATH_INFO: `/v3/projects/2e273d570dd044dcacd7b1293025ed46` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:49.509602 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fc0bc5-e2aa-4532-bcaf-00ec9d63ab2f admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e273d570dd044dcacd7b1293025ed46)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:49.511532 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fc0bc5-e2aa-4532-bcaf-00ec9d63ab2f admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:49.544142 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:51:49.562752 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-10fc0bc5-e2aa-4532-bcaf-00ec9d63ab2f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e273d570dd044dcacd7b1293025ed46', 'request_id': 'req-10fc0bc5-e2aa-4532-bcaf-00ec9d63ab2f'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:49.571468 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-10fc0bc5-e2aa-4532-bcaf-00ec9d63ab2f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e273d570dd044dcacd7b1293025ed46', 'request_id': 'req-10fc0bc5-e2aa-4532-bcaf-00ec9d63ab2f'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:49.577290 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-10fc0bc5-e2aa-4532-bcaf-00ec9d63ab2f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e273d570dd044dcacd7b1293025ed46', 'request_id': 'req-10fc0bc5-e2aa-4532-bcaf-00ec9d63ab2f'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:49.589408 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-10fc0bc5-e2aa-4532-bcaf-00ec9d63ab2f admin admin] The token cache is being invalidate because project 2e273d570dd044dcacd7b1293025ed46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80357) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:51:49.589695 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-10fc0bc5-e2aa-4532-bcaf-00ec9d63ab2f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:49.590330 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-10fc0bc5-e2aa-4532-bcaf-00ec9d63ab2f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:49.592116 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1326/2672] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jun 16 17:51:49 2022] DELETE /identity/v3/projects/2e273d570dd044dcacd7b1293025ed46 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:51:49.596800 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e45e73-9af4-4867-b271-a94f3ceeb4a1 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:49.597199 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e45e73-9af4-4867-b271-a94f3ceeb4a1 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:49.598114 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e45e73-9af4-4867-b271-a94f3ceeb4a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:49.618966 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-95e45e73-9af4-4867-b271-a94f3ceeb4a1 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:51:49.620140 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-95e45e73-9af4-4867-b271-a94f3ceeb4a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:51:49.689333 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1347/2673] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:51:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:51:49.697082 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a58bbf-d21c-4a7f-b73a-f466f74111c1 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:49.697378 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a58bbf-d21c-4a7f-b73a-f466f74111c1 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:49.697806 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a58bbf-d21c-4a7f-b73a-f466f74111c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:49.717198 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-47a58bbf-d21c-4a7f-b73a-f466f74111c1 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:51:49.718475 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-47a58bbf-d21c-4a7f-b73a-f466f74111c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:51:49.729470 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1327/2674] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:51:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:51:49.733798 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e45e73-9af4-4867-b271-a94f3ceeb4a1 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:49.747970 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e45e73-9af4-4867-b271-a94f3ceeb4a1 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:49.753219 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a04b9e-f35d-4148-8a1c-2c5f7e05bae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:49.754095 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a04b9e-f35d-4148-8a1c-2c5f7e05bae3 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:49.754374 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a04b9e-f35d-4148-8a1c-2c5f7e05bae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:49.754655 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a04b9e-f35d-4148-8a1c-2c5f7e05bae3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:49.755380 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a04b9e-f35d-4148-8a1c-2c5f7e05bae3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:51:49.755684 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a04b9e-f35d-4148-8a1c-2c5f7e05bae3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:49.757309 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a04b9e-f35d-4148-8a1c-2c5f7e05bae3 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:49.773746 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1348/2675] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jun 16 17:51:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:51:49.776011 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a58bbf-d21c-4a7f-b73a-f466f74111c1 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:49.787418 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a58bbf-d21c-4a7f-b73a-f466f74111c1 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:49.792183 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfec9b4-ad76-49a5-99d1-3b79646a371a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:49.792871 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfec9b4-ad76-49a5-99d1-3b79646a371a admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:49.793277 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfec9b4-ad76-49a5-99d1-3b79646a371a admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:49.793557 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfec9b4-ad76-49a5-99d1-3b79646a371a admin admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:49.794241 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfec9b4-ad76-49a5-99d1-3b79646a371a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:49.796162 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfec9b4-ad76-49a5-99d1-3b79646a371a admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:49.824939 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1328/2676] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jun 16 17:51:49 2022] POST /identity/v3/projects => generated 387 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:51:49.829101 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a04b9e-f35d-4148-8a1c-2c5f7e05bae3 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:49.840131 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a04b9e-f35d-4148-8a1c-2c5f7e05bae3 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:49.876925 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:51:49.908031 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16654bbe-da45-43f0-9cdc-829e4dd866d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:49.908728 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16654bbe-da45-43f0-9cdc-829e4dd866d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:49.909032 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16654bbe-da45-43f0-9cdc-829e4dd866d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:49.909307 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16654bbe-da45-43f0-9cdc-829e4dd866d8 admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:49.910098 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16654bbe-da45-43f0-9cdc-829e4dd866d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:49.912644 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16654bbe-da45-43f0-9cdc-829e4dd866d8 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:49.934165 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1349/2677] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jun 16 17:51:49 2022] POST /identity/v3/users => generated 294 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:51:49.938173 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfec9b4-ad76-49a5-99d1-3b79646a371a admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:49.949071 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfec9b4-ad76-49a5-99d1-3b79646a371a admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:49.963129 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d273a29-33c0-4eff-baba-72e520c0521c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:49.963397 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d273a29-33c0-4eff-baba-72e520c0521c admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:49.963688 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d273a29-33c0-4eff-baba-72e520c0521c admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:49.963968 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d273a29-33c0-4eff-baba-72e520c0521c admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:49.964774 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d273a29-33c0-4eff-baba-72e520c0521c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:49.966467 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d273a29-33c0-4eff-baba-72e520c0521c admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:49.980355 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1329/2678] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:51:49 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:51:49.984454 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16654bbe-da45-43f0-9cdc-829e4dd866d8 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:49.995342 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16654bbe-da45-43f0-9cdc-829e4dd866d8 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:50.000289 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd1943e-3574-45c1-911f-328e84fca6ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:50.001018 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd1943e-3574-45c1-911f-328e84fca6ab admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:50.001298 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd1943e-3574-45c1-911f-328e84fca6ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:50.001625 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd1943e-3574-45c1-911f-328e84fca6ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:50.002362 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd1943e-3574-45c1-911f-328e84fca6ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:50.004143 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd1943e-3574-45c1-911f-328e84fca6ab admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:50.018651 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1350/2679] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:51:49 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:51:50.024021 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d273a29-33c0-4eff-baba-72e520c0521c admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:50.037342 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d273a29-33c0-4eff-baba-72e520c0521c admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:50.044280 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f1302c-9c4e-4cdb-ac54-b47020898b99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:50.045328 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f1302c-9c4e-4cdb-ac54-b47020898b99 admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:50.045620 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f1302c-9c4e-4cdb-ac54-b47020898b99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:50.046004 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f1302c-9c4e-4cdb-ac54-b47020898b99 admin admin] PATH_INFO: `/v3/projects/d6ab13ef23d44186bcdb1975eb6ab95b/users/23ccf3fa34584cef885ed3839571d3f0/roles/0be2075946f54c8485f1c275f746478a` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:50.065546 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f1302c-9c4e-4cdb-ac54-b47020898b99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6ab13ef23d44186bcdb1975eb6ab95b, user_id=23ccf3fa34584cef885ed3839571d3f0, role_id=0be2075946f54c8485f1c275f746478a)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:50.069394 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f1302c-9c4e-4cdb-ac54-b47020898b99 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:50.084661 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1330/2680] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:51:50 2022] PUT /identity/v3/projects/d6ab13ef23d44186bcdb1975eb6ab95b/users/23ccf3fa34584cef885ed3839571d3f0/roles/0be2075946f54c8485f1c275f746478a => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:51:50.095088 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f5ca05-bfd6-4272-9971-c2c71312c554 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:50.095912 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f5ca05-bfd6-4272-9971-c2c71312c554 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:50.096400 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f5ca05-bfd6-4272-9971-c2c71312c554 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:50.131839 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-05f5ca05-bfd6-4272-9971-c2c71312c554 None None] MFA Rules not processed for user `23ccf3fa34584cef885ed3839571d3f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:51:50.132425 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-05f5ca05-bfd6-4272-9971-c2c71312c554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:51:50.223964 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1351/2681] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:51:50 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:51:50.232892 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f1302c-9c4e-4cdb-ac54-b47020898b99 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:50.311818 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f1302c-9c4e-4cdb-ac54-b47020898b99 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:50.316592 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8476b1-98ab-44ba-9509-a74fdd0ed959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:50.317256 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8476b1-98ab-44ba-9509-a74fdd0ed959 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:50.317544 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8476b1-98ab-44ba-9509-a74fdd0ed959 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:50.317822 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8476b1-98ab-44ba-9509-a74fdd0ed959 admin admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:50.318554 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8476b1-98ab-44ba-9509-a74fdd0ed959 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:50.320520 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8476b1-98ab-44ba-9509-a74fdd0ed959 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:50.348815 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1331/2682] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jun 16 17:51:50 2022] POST /identity/v3/projects => generated 385 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:51:50.352821 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f5ca05-bfd6-4272-9971-c2c71312c554 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:50.364248 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f5ca05-bfd6-4272-9971-c2c71312c554 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:50.423658 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a430da-b324-4892-9e47-74b770fb4a5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:50.424330 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a430da-b324-4892-9e47-74b770fb4a5b admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:50.424654 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a430da-b324-4892-9e47-74b770fb4a5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:50.424938 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a430da-b324-4892-9e47-74b770fb4a5b admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:50.430016 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a430da-b324-4892-9e47-74b770fb4a5b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:50.430016 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a430da-b324-4892-9e47-74b770fb4a5b admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:50.450662 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1352/2683] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jun 16 17:51:50 2022] POST /identity/v3/users => generated 299 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:51:50.455297 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8476b1-98ab-44ba-9509-a74fdd0ed959 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:50.468163 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8476b1-98ab-44ba-9509-a74fdd0ed959 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:50.474115 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ffb8c5-c769-4313-bed5-5b5f992e99df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:50.474788 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ffb8c5-c769-4313-bed5-5b5f992e99df admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:50.475075 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ffb8c5-c769-4313-bed5-5b5f992e99df admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:50.475350 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ffb8c5-c769-4313-bed5-5b5f992e99df admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:50.476084 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ffb8c5-c769-4313-bed5-5b5f992e99df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:50.477890 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ffb8c5-c769-4313-bed5-5b5f992e99df admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:50.492411 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1332/2684] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:51:50 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:51:50.494491 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a430da-b324-4892-9e47-74b770fb4a5b admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:50.504894 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a430da-b324-4892-9e47-74b770fb4a5b admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:50.509783 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b968e26f-7c0d-4416-ab4e-b69afea3bd83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:50.510661 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b968e26f-7c0d-4416-ab4e-b69afea3bd83 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:50.510947 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b968e26f-7c0d-4416-ab4e-b69afea3bd83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:50.511232 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b968e26f-7c0d-4416-ab4e-b69afea3bd83 admin admin] PATH_INFO: `/v3/projects/e08c4a5e8a3f454eb84d8ea93db97816/users/c5d4ca6f00084863acff3ef3c2744215/roles/3ca2dcbbd29d4a31a3ab78f2299bc239` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:50.527742 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b968e26f-7c0d-4416-ab4e-b69afea3bd83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e08c4a5e8a3f454eb84d8ea93db97816, user_id=c5d4ca6f00084863acff3ef3c2744215, role_id=3ca2dcbbd29d4a31a3ab78f2299bc239)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:50.530595 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b968e26f-7c0d-4416-ab4e-b69afea3bd83 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:50.545502 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1353/2685] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:51:50 2022] PUT /identity/v3/projects/e08c4a5e8a3f454eb84d8ea93db97816/users/c5d4ca6f00084863acff3ef3c2744215/roles/3ca2dcbbd29d4a31a3ab78f2299bc239 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:51:50.549348 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ffb8c5-c769-4313-bed5-5b5f992e99df admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:50.560401 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ffb8c5-c769-4313-bed5-5b5f992e99df admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:50.583930 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:51:50.634367 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621b4c81-6f28-463c-9ca3-5a6165cb04a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:50.635419 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621b4c81-6f28-463c-9ca3-5a6165cb04a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:50.635797 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621b4c81-6f28-463c-9ca3-5a6165cb04a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:50.636221 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621b4c81-6f28-463c-9ca3-5a6165cb04a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:50.637654 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621b4c81-6f28-463c-9ca3-5a6165cb04a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:50.640194 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621b4c81-6f28-463c-9ca3-5a6165cb04a1 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:50.656818 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1333/2686] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:51:50 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:51:50.661425 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b968e26f-7c0d-4416-ab4e-b69afea3bd83 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:50.674183 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b968e26f-7c0d-4416-ab4e-b69afea3bd83 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:50.681809 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9113b4-4dab-4c2c-9f75-52940b6169e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:50.682730 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9113b4-4dab-4c2c-9f75-52940b6169e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:50.683190 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9113b4-4dab-4c2c-9f75-52940b6169e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:50.683551 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9113b4-4dab-4c2c-9f75-52940b6169e4 admin admin] PATH_INFO: `/v3/projects/e08c4a5e8a3f454eb84d8ea93db97816/users/c5d4ca6f00084863acff3ef3c2744215/roles/3ca2dcbbd29d4a31a3ab78f2299bc239` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:50.685654 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9113b4-4dab-4c2c-9f75-52940b6169e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e08c4a5e8a3f454eb84d8ea93db97816, user_id=c5d4ca6f00084863acff3ef3c2744215, role_id=3ca2dcbbd29d4a31a3ab78f2299bc239)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:50.689110 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9113b4-4dab-4c2c-9f75-52940b6169e4 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:50.696679 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:51:50.705026 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1354/2687] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:51:50 2022] PUT /identity/v3/projects/e08c4a5e8a3f454eb84d8ea93db97816/users/c5d4ca6f00084863acff3ef3c2744215/roles/3ca2dcbbd29d4a31a3ab78f2299bc239 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:51:50.717514 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f3e88c-663c-4858-80a8-cf61ec697c03 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:50.717941 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f3e88c-663c-4858-80a8-cf61ec697c03 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:50.718090 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f3e88c-663c-4858-80a8-cf61ec697c03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:50.750782 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-88f3e88c-663c-4858-80a8-cf61ec697c03 None None] MFA Rules not processed for user `c5d4ca6f00084863acff3ef3c2744215`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:51:50.750782 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-88f3e88c-663c-4858-80a8-cf61ec697c03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:51:50.826990 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1334/2688] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:51:50 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:51:50.838760 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9113b4-4dab-4c2c-9f75-52940b6169e4 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:50.840513 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-5d9113b4-4dab-4c2c-9f75-52940b6169e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:51:50.900285 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9113b4-4dab-4c2c-9f75-52940b6169e4 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:50.904228 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e2829f-3390-4980-a629-6ea7892b6807 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24d5e2637b7043e08313909d5b06e329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:50.904906 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e2829f-3390-4980-a629-6ea7892b6807 service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:50.905196 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e2829f-3390-4980-a629-6ea7892b6807 service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:50.905478 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e2829f-3390-4980-a629-6ea7892b6807 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:50.915632 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e2829f-3390-4980-a629-6ea7892b6807 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:50.917137 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e2829f-3390-4980-a629-6ea7892b6807 service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:50.926663 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1355/2689] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:51:50 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:51:51.516070 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f3e88c-663c-4858-80a8-cf61ec697c03 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:51.584407 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f3e88c-663c-4858-80a8-cf61ec697c03 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:51.589582 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b20d847-8279-42aa-85c3-3810fdc88dfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:51.590505 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b20d847-8279-42aa-85c3-3810fdc88dfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:51.590784 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b20d847-8279-42aa-85c3-3810fdc88dfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:51.591207 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b20d847-8279-42aa-85c3-3810fdc88dfe admin admin] PATH_INFO: `/v3/users/23ccf3fa34584cef885ed3839571d3f0` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:51.592425 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b20d847-8279-42aa-85c3-3810fdc88dfe admin admin] RBAC: Authorizing `identity:delete_user(user_id=23ccf3fa34584cef885ed3839571d3f0)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:51.594332 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b20d847-8279-42aa-85c3-3810fdc88dfe admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:51.647179 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-1b20d847-8279-42aa-85c3-3810fdc88dfe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23ccf3fa34584cef885ed3839571d3f0'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:51.647926 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-1b20d847-8279-42aa-85c3-3810fdc88dfe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23ccf3fa34584cef885ed3839571d3f0'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:51.649724 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:51:51.658317 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-1b20d847-8279-42aa-85c3-3810fdc88dfe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23ccf3fa34584cef885ed3839571d3f0'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:51.659050 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-1b20d847-8279-42aa-85c3-3810fdc88dfe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23ccf3fa34584cef885ed3839571d3f0'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:51.672234 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-1b20d847-8279-42aa-85c3-3810fdc88dfe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23ccf3fa34584cef885ed3839571d3f0'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:51.685081 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1335/2690] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jun 16 17:51:51 2022] DELETE /identity/v3/users/23ccf3fa34584cef885ed3839571d3f0 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:51:51.689616 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a9016c-47de-4860-9888-3e72beec1163 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:51.690100 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a9016c-47de-4860-9888-3e72beec1163 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:51.690221 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a9016c-47de-4860-9888-3e72beec1163 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:51.710297 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-d1a9016c-47de-4860-9888-3e72beec1163 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:51:51.711511 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-d1a9016c-47de-4860-9888-3e72beec1163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:51:51.773297 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1356/2691] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:51:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:51:51.786400 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b20d847-8279-42aa-85c3-3810fdc88dfe admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:51.787885 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-1b20d847-8279-42aa-85c3-3810fdc88dfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:51:51.851471 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b20d847-8279-42aa-85c3-3810fdc88dfe admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:51.855114 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65597f6-ab96-46ca-aa33-cc989d25c8a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:51.855758 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65597f6-ab96-46ca-aa33-cc989d25c8a3 service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:51.856044 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65597f6-ab96-46ca-aa33-cc989d25c8a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:51.856321 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65597f6-ab96-46ca-aa33-cc989d25c8a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:51.865538 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65597f6-ab96-46ca-aa33-cc989d25c8a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:51.866975 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65597f6-ab96-46ca-aa33-cc989d25c8a3 service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:51.876707 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1336/2692] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:51:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 17:51:52.152579 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a9016c-47de-4860-9888-3e72beec1163 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:52.154027 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-d1a9016c-47de-4860-9888-3e72beec1163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:51:52.166548 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a9016c-47de-4860-9888-3e72beec1163 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:52.171280 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a161c46-0ba1-420b-9020-00d95a611748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:52.172142 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a161c46-0ba1-420b-9020-00d95a611748 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:52.172466 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a161c46-0ba1-420b-9020-00d95a611748 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:52.172747 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a161c46-0ba1-420b-9020-00d95a611748 admin admin] PATH_INFO: `/v3/projects/d6ab13ef23d44186bcdb1975eb6ab95b` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:52.173939 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a161c46-0ba1-420b-9020-00d95a611748 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6ab13ef23d44186bcdb1975eb6ab95b)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:52.176356 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a161c46-0ba1-420b-9020-00d95a611748 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:52.226873 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-8a161c46-0ba1-420b-9020-00d95a611748 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6ab13ef23d44186bcdb1975eb6ab95b', 'request_id': 'req-8a161c46-0ba1-420b-9020-00d95a611748'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:52.235233 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-8a161c46-0ba1-420b-9020-00d95a611748 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6ab13ef23d44186bcdb1975eb6ab95b', 'request_id': 'req-8a161c46-0ba1-420b-9020-00d95a611748'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:52.242086 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-8a161c46-0ba1-420b-9020-00d95a611748 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6ab13ef23d44186bcdb1975eb6ab95b', 'request_id': 'req-8a161c46-0ba1-420b-9020-00d95a611748'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:52.252307 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-8a161c46-0ba1-420b-9020-00d95a611748 admin admin] The token cache is being invalidate because project d6ab13ef23d44186bcdb1975eb6ab95b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:51:52.252632 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-8a161c46-0ba1-420b-9020-00d95a611748 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:52.253266 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-8a161c46-0ba1-420b-9020-00d95a611748 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:52.255323 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1357/2693] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jun 16 17:51:52 2022] DELETE /identity/v3/projects/d6ab13ef23d44186bcdb1975eb6ab95b => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:51:52.259322 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65597f6-ab96-46ca-aa33-cc989d25c8a3 service neutron] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:52.269323 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65597f6-ab96-46ca-aa33-cc989d25c8a3 service neutron] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:52.326843 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ac0f37-c0ce-48ae-a59c-f3c1e99fd2d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:52.327705 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ac0f37-c0ce-48ae-a59c-f3c1e99fd2d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:52.327990 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ac0f37-c0ce-48ae-a59c-f3c1e99fd2d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:52.328269 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ac0f37-c0ce-48ae-a59c-f3c1e99fd2d1 admin admin] PATH_INFO: `/v3/users/c5d4ca6f00084863acff3ef3c2744215` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:52.329477 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ac0f37-c0ce-48ae-a59c-f3c1e99fd2d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5d4ca6f00084863acff3ef3c2744215)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:52.331393 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ac0f37-c0ce-48ae-a59c-f3c1e99fd2d1 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:52.376655 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-a9ac0f37-c0ce-48ae-a59c-f3c1e99fd2d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5d4ca6f00084863acff3ef3c2744215'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:52.377304 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-a9ac0f37-c0ce-48ae-a59c-f3c1e99fd2d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5d4ca6f00084863acff3ef3c2744215'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:52.384812 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-a9ac0f37-c0ce-48ae-a59c-f3c1e99fd2d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5d4ca6f00084863acff3ef3c2744215'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:52.385457 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-a9ac0f37-c0ce-48ae-a59c-f3c1e99fd2d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5d4ca6f00084863acff3ef3c2744215'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:52.398189 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:51:52.399772 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-a9ac0f37-c0ce-48ae-a59c-f3c1e99fd2d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5d4ca6f00084863acff3ef3c2744215'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:52.415424 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1337/2694] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jun 16 17:51:52 2022] DELETE /identity/v3/users/c5d4ca6f00084863acff3ef3c2744215 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:51:52.699306 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a161c46-0ba1-420b-9020-00d95a611748 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:52.700973 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-8a161c46-0ba1-420b-9020-00d95a611748 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:51:52.719535 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a161c46-0ba1-420b-9020-00d95a611748 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:52.743208 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:51:52.782424 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ebcdc4-7990-4d51-8421-b223b951c052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:52.783209 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ebcdc4-7990-4d51-8421-b223b951c052 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:52.783499 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ebcdc4-7990-4d51-8421-b223b951c052 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:52.783777 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ebcdc4-7990-4d51-8421-b223b951c052 admin admin] PATH_INFO: `/v3/projects/e08c4a5e8a3f454eb84d8ea93db97816` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:52.786150 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ebcdc4-7990-4d51-8421-b223b951c052 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e08c4a5e8a3f454eb84d8ea93db97816)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:52.788045 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ebcdc4-7990-4d51-8421-b223b951c052 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:52.835896 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-c3ebcdc4-7990-4d51-8421-b223b951c052 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e08c4a5e8a3f454eb84d8ea93db97816', 'request_id': 'req-c3ebcdc4-7990-4d51-8421-b223b951c052'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:52.843824 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-c3ebcdc4-7990-4d51-8421-b223b951c052 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e08c4a5e8a3f454eb84d8ea93db97816', 'request_id': 'req-c3ebcdc4-7990-4d51-8421-b223b951c052'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:52.851307 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-c3ebcdc4-7990-4d51-8421-b223b951c052 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e08c4a5e8a3f454eb84d8ea93db97816', 'request_id': 'req-c3ebcdc4-7990-4d51-8421-b223b951c052'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:52.862369 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-c3ebcdc4-7990-4d51-8421-b223b951c052 admin admin] The token cache is being invalidate because project e08c4a5e8a3f454eb84d8ea93db97816 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:51:52.862663 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-c3ebcdc4-7990-4d51-8421-b223b951c052 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:52.864595 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-c3ebcdc4-7990-4d51-8421-b223b951c052 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:52.866696 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1358/2695] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jun 16 17:51:52 2022] DELETE /identity/v3/projects/e08c4a5e8a3f454eb84d8ea93db97816 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:51:52.871241 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fefce4-f8cb-41ee-9058-b0ffa6969dfe None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:52.871711 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fefce4-f8cb-41ee-9058-b0ffa6969dfe None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:52.871807 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fefce4-f8cb-41ee-9058-b0ffa6969dfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:52.890719 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-b5fefce4-f8cb-41ee-9058-b0ffa6969dfe None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:51:52.891918 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-b5fefce4-f8cb-41ee-9058-b0ffa6969dfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:51:52.953809 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1338/2696] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:51:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:51:52.962251 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cb730c-5e74-40fc-b5e0-d3a04be85b3e None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:52.962549 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cb730c-5e74-40fc-b5e0-d3a04be85b3e None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:52.962837 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cb730c-5e74-40fc-b5e0-d3a04be85b3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:52.981760 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-99cb730c-5e74-40fc-b5e0-d3a04be85b3e None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:51:52.983011 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-99cb730c-5e74-40fc-b5e0-d3a04be85b3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:51:52.991486 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1359/2697] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:51:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:51:53.003398 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fefce4-f8cb-41ee-9058-b0ffa6969dfe None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:53.013441 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fefce4-f8cb-41ee-9058-b0ffa6969dfe None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:53.018292 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43491e0d-23d1-48cd-89be-3471bc2c21c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:53.018968 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43491e0d-23d1-48cd-89be-3471bc2c21c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:53.019777 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43491e0d-23d1-48cd-89be-3471bc2c21c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:53.020134 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43491e0d-23d1-48cd-89be-3471bc2c21c2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:53.021177 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43491e0d-23d1-48cd-89be-3471bc2c21c2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:51:53.021481 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43491e0d-23d1-48cd-89be-3471bc2c21c2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:53.023098 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43491e0d-23d1-48cd-89be-3471bc2c21c2 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:53.038227 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1339/2698] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jun 16 17:51:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:51:53.044840 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cb730c-5e74-40fc-b5e0-d3a04be85b3e None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:53.054994 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cb730c-5e74-40fc-b5e0-d3a04be85b3e None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:53.059615 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7263f0-1925-4453-8e1e-6824e332feaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:53.060263 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7263f0-1925-4453-8e1e-6824e332feaa admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:53.060583 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7263f0-1925-4453-8e1e-6824e332feaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:53.060855 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7263f0-1925-4453-8e1e-6824e332feaa admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:53.061662 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7263f0-1925-4453-8e1e-6824e332feaa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:53.063554 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7263f0-1925-4453-8e1e-6824e332feaa admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:53.088832 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1360/2699] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jun 16 17:51:53 2022] POST /identity/v3/projects => generated 381 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:51:53.093019 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43491e0d-23d1-48cd-89be-3471bc2c21c2 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:53.103773 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43491e0d-23d1-48cd-89be-3471bc2c21c2 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:53.161156 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0475589a-4526-4fd5-ad24-1d2173c761b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:53.161862 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0475589a-4526-4fd5-ad24-1d2173c761b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:53.162149 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0475589a-4526-4fd5-ad24-1d2173c761b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:53.162456 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0475589a-4526-4fd5-ad24-1d2173c761b4 admin admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:53.163135 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0475589a-4526-4fd5-ad24-1d2173c761b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:53.165125 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0475589a-4526-4fd5-ad24-1d2173c761b4 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:53.183806 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1340/2700] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jun 16 17:51:53 2022] POST /identity/v3/users => generated 291 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:51:53.187850 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7263f0-1925-4453-8e1e-6824e332feaa admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:53.198178 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7263f0-1925-4453-8e1e-6824e332feaa admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:53.202883 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b3ea7b-a5e8-4e59-a203-48a3cf90dc8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:53.203557 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b3ea7b-a5e8-4e59-a203-48a3cf90dc8e admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:53.203872 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b3ea7b-a5e8-4e59-a203-48a3cf90dc8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:53.204154 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b3ea7b-a5e8-4e59-a203-48a3cf90dc8e admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:53.204944 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b3ea7b-a5e8-4e59-a203-48a3cf90dc8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:53.206579 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b3ea7b-a5e8-4e59-a203-48a3cf90dc8e admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:53.217606 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1361/2701] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:51:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 17:51:53.221367 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0475589a-4526-4fd5-ad24-1d2173c761b4 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:53.233201 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0475589a-4526-4fd5-ad24-1d2173c761b4 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:53.237877 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6201b8ef-e90a-4b27-8d5a-93ef2b293ff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:53.238827 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6201b8ef-e90a-4b27-8d5a-93ef2b293ff4 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:53.239117 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6201b8ef-e90a-4b27-8d5a-93ef2b293ff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:53.239400 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6201b8ef-e90a-4b27-8d5a-93ef2b293ff4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:53.240132 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6201b8ef-e90a-4b27-8d5a-93ef2b293ff4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:53.241763 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6201b8ef-e90a-4b27-8d5a-93ef2b293ff4 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:53.251650 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1341/2702] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:51:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:51:53.256569 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b3ea7b-a5e8-4e59-a203-48a3cf90dc8e admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:53.266741 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b3ea7b-a5e8-4e59-a203-48a3cf90dc8e admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:53.272055 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4234a07a-528b-4779-ba6d-7c79b4848f65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:53.272957 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4234a07a-528b-4779-ba6d-7c79b4848f65 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:53.273238 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4234a07a-528b-4779-ba6d-7c79b4848f65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:53.273510 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4234a07a-528b-4779-ba6d-7c79b4848f65 admin admin] PATH_INFO: `/v3/projects/44f228226e624e5589683d0cd27e9374/users/7239eff625554ee99207953c70cac991/roles/0be2075946f54c8485f1c275f746478a` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:53.288543 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4234a07a-528b-4779-ba6d-7c79b4848f65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44f228226e624e5589683d0cd27e9374, user_id=7239eff625554ee99207953c70cac991, role_id=0be2075946f54c8485f1c275f746478a)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:53.291271 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4234a07a-528b-4779-ba6d-7c79b4848f65 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:53.305015 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1362/2703] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:51:53 2022] PUT /identity/v3/projects/44f228226e624e5589683d0cd27e9374/users/7239eff625554ee99207953c70cac991/roles/0be2075946f54c8485f1c275f746478a => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:51:53.316219 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd0e15a-cf93-463c-af1a-5f86470bed62 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:53.316594 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd0e15a-cf93-463c-af1a-5f86470bed62 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:53.316966 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd0e15a-cf93-463c-af1a-5f86470bed62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:53.343389 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-bfd0e15a-cf93-463c-af1a-5f86470bed62 None None] MFA Rules not processed for user `7239eff625554ee99207953c70cac991`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:51:53.344620 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-bfd0e15a-cf93-463c-af1a-5f86470bed62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:51:53.388932 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:51:53.412953 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1342/2704] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:51:53 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:51:53.417366 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4234a07a-528b-4779-ba6d-7c79b4848f65 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:53.470197 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:51:53.483825 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4234a07a-528b-4779-ba6d-7c79b4848f65 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:53.491221 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb308215-bc23-4c3f-ab02-84cf93651b91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:53.491896 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb308215-bc23-4c3f-ab02-84cf93651b91 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:53.492187 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb308215-bc23-4c3f-ab02-84cf93651b91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:53.492502 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb308215-bc23-4c3f-ab02-84cf93651b91 admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:53.493249 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb308215-bc23-4c3f-ab02-84cf93651b91 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:53.495254 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb308215-bc23-4c3f-ab02-84cf93651b91 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:53.524034 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1363/2705] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jun 16 17:51:53 2022] POST /identity/v3/projects => generated 379 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:51:53.528212 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd0e15a-cf93-463c-af1a-5f86470bed62 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:53.540264 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd0e15a-cf93-463c-af1a-5f86470bed62 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:53.602699 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234e44b5-6ba5-45be-beb1-8eafe2a9c411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:53.603467 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234e44b5-6ba5-45be-beb1-8eafe2a9c411 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:53.603782 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234e44b5-6ba5-45be-beb1-8eafe2a9c411 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:53.604118 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234e44b5-6ba5-45be-beb1-8eafe2a9c411 admin admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:53.604817 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234e44b5-6ba5-45be-beb1-8eafe2a9c411 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:53.607019 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234e44b5-6ba5-45be-beb1-8eafe2a9c411 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:53.633063 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1343/2706] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jun 16 17:51:53 2022] POST /identity/v3/users => generated 296 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:51:53.634571 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb308215-bc23-4c3f-ab02-84cf93651b91 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:53.646647 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb308215-bc23-4c3f-ab02-84cf93651b91 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:53.651381 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59691377-7060-4bfa-bee9-2ec99fd48a03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:53.652072 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59691377-7060-4bfa-bee9-2ec99fd48a03 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:53.652358 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59691377-7060-4bfa-bee9-2ec99fd48a03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:53.652662 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59691377-7060-4bfa-bee9-2ec99fd48a03 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:53.653422 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59691377-7060-4bfa-bee9-2ec99fd48a03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:53.655052 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59691377-7060-4bfa-bee9-2ec99fd48a03 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:53.667856 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1364/2707] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:51:53 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:51:53.671940 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234e44b5-6ba5-45be-beb1-8eafe2a9c411 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:53.682067 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234e44b5-6ba5-45be-beb1-8eafe2a9c411 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:53.687075 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf0cca0-7dcc-4f32-98e6-aca1e5ccf952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:53.687965 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf0cca0-7dcc-4f32-98e6-aca1e5ccf952 admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:53.688248 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf0cca0-7dcc-4f32-98e6-aca1e5ccf952 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:53.688561 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf0cca0-7dcc-4f32-98e6-aca1e5ccf952 admin admin] PATH_INFO: `/v3/projects/e597a2041fe54a7b8a0719367f668511/users/fbd0eae38a3b4545a0d1559731ee6982/roles/3ca2dcbbd29d4a31a3ab78f2299bc239` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:53.704604 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf0cca0-7dcc-4f32-98e6-aca1e5ccf952 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e597a2041fe54a7b8a0719367f668511, user_id=fbd0eae38a3b4545a0d1559731ee6982, role_id=3ca2dcbbd29d4a31a3ab78f2299bc239)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:53.708492 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf0cca0-7dcc-4f32-98e6-aca1e5ccf952 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:53.722370 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1344/2708] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:51:53 2022] PUT /identity/v3/projects/e597a2041fe54a7b8a0719367f668511/users/fbd0eae38a3b4545a0d1559731ee6982/roles/3ca2dcbbd29d4a31a3ab78f2299bc239 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:51:53.726643 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59691377-7060-4bfa-bee9-2ec99fd48a03 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:53.737073 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59691377-7060-4bfa-bee9-2ec99fd48a03 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:53.799524 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3044d7-28da-4ad1-9468-deda4261b06c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:53.800127 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3044d7-28da-4ad1-9468-deda4261b06c admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:53.800982 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3044d7-28da-4ad1-9468-deda4261b06c admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:53.801267 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3044d7-28da-4ad1-9468-deda4261b06c admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:53.802003 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3044d7-28da-4ad1-9468-deda4261b06c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:53.803617 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3044d7-28da-4ad1-9468-deda4261b06c admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:53.814245 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1365/2709] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:51:53 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:51:53.819645 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf0cca0-7dcc-4f32-98e6-aca1e5ccf952 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:53.830521 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf0cca0-7dcc-4f32-98e6-aca1e5ccf952 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:53.836794 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4fdb4e-0ddb-463f-8072-f32abffc93dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:53.837702 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4fdb4e-0ddb-463f-8072-f32abffc93dc admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:53.838060 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4fdb4e-0ddb-463f-8072-f32abffc93dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:53.838408 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4fdb4e-0ddb-463f-8072-f32abffc93dc admin admin] PATH_INFO: `/v3/projects/e597a2041fe54a7b8a0719367f668511/users/fbd0eae38a3b4545a0d1559731ee6982/roles/3ca2dcbbd29d4a31a3ab78f2299bc239` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:53.841199 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4fdb4e-0ddb-463f-8072-f32abffc93dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=e597a2041fe54a7b8a0719367f668511, user_id=fbd0eae38a3b4545a0d1559731ee6982, role_id=3ca2dcbbd29d4a31a3ab78f2299bc239)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:53.844006 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4fdb4e-0ddb-463f-8072-f32abffc93dc admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:53.857170 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1345/2710] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:51:53 2022] PUT /identity/v3/projects/e597a2041fe54a7b8a0719367f668511/users/fbd0eae38a3b4545a0d1559731ee6982/roles/3ca2dcbbd29d4a31a3ab78f2299bc239 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:51:53.868122 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dcf6ab-9c72-4623-aedf-09dd79c4ebf6 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:53.868446 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dcf6ab-9c72-4623-aedf-09dd79c4ebf6 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:53.868731 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dcf6ab-9c72-4623-aedf-09dd79c4ebf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:53.899897 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-19dcf6ab-9c72-4623-aedf-09dd79c4ebf6 None None] MFA Rules not processed for user `fbd0eae38a3b4545a0d1559731ee6982`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:51:53.901223 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-19dcf6ab-9c72-4623-aedf-09dd79c4ebf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:51:53.976432 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1366/2711] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:51:53 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:51:53.988612 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4fdb4e-0ddb-463f-8072-f32abffc93dc admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:53.990461 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-6b4fdb4e-0ddb-463f-8072-f32abffc93dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:51:54.053565 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4fdb4e-0ddb-463f-8072-f32abffc93dc admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:54.057464 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1448ea-b112-493b-b4b8-362d1cf02087 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24d5e2637b7043e08313909d5b06e329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:54.058120 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1448ea-b112-493b-b4b8-362d1cf02087 service cinder] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:54.058609 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1448ea-b112-493b-b4b8-362d1cf02087 service cinder] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:54.058732 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1448ea-b112-493b-b4b8-362d1cf02087 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:54.068609 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1448ea-b112-493b-b4b8-362d1cf02087 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:54.070042 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1448ea-b112-493b-b4b8-362d1cf02087 service cinder] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:54.079600 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1346/2712] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:51:53 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:51:54.456653 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dcf6ab-9c72-4623-aedf-09dd79c4ebf6 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:54.505369 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:51:54.521558 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dcf6ab-9c72-4623-aedf-09dd79c4ebf6 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:54.526192 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19566258-ad5d-4aa3-9bb8-6e393fb3e2cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:54.527078 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19566258-ad5d-4aa3-9bb8-6e393fb3e2cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:54.527345 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19566258-ad5d-4aa3-9bb8-6e393fb3e2cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:54.527622 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19566258-ad5d-4aa3-9bb8-6e393fb3e2cf admin admin] PATH_INFO: `/v3/users/7239eff625554ee99207953c70cac991` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:54.528861 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19566258-ad5d-4aa3-9bb8-6e393fb3e2cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=7239eff625554ee99207953c70cac991)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:54.530789 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19566258-ad5d-4aa3-9bb8-6e393fb3e2cf admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:54.582105 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-19566258-ad5d-4aa3-9bb8-6e393fb3e2cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7239eff625554ee99207953c70cac991'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:54.595028 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-19566258-ad5d-4aa3-9bb8-6e393fb3e2cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7239eff625554ee99207953c70cac991'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:54.595712 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-19566258-ad5d-4aa3-9bb8-6e393fb3e2cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7239eff625554ee99207953c70cac991'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:54.603318 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-19566258-ad5d-4aa3-9bb8-6e393fb3e2cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7239eff625554ee99207953c70cac991'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:54.610682 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-19566258-ad5d-4aa3-9bb8-6e393fb3e2cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7239eff625554ee99207953c70cac991'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:54.617264 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1367/2713] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jun 16 17:51:54 2022] DELETE /identity/v3/users/7239eff625554ee99207953c70cac991 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:51:54.621716 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d4cc4b-e147-4f1c-b7d3-1f529fb5fb35 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:54.622002 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d4cc4b-e147-4f1c-b7d3-1f529fb5fb35 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:54.622314 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d4cc4b-e147-4f1c-b7d3-1f529fb5fb35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:54.641605 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-57d4cc4b-e147-4f1c-b7d3-1f529fb5fb35 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:51:54.642808 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-57d4cc4b-e147-4f1c-b7d3-1f529fb5fb35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:51:54.676585 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:51:54.703979 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1347/2714] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:51:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:51:54.714215 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19566258-ad5d-4aa3-9bb8-6e393fb3e2cf admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:54.715692 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-19566258-ad5d-4aa3-9bb8-6e393fb3e2cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:51:54.782007 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19566258-ad5d-4aa3-9bb8-6e393fb3e2cf admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:54.786262 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2e02ea-61a3-4ce2-b979-b68c935be87f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:54.786915 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2e02ea-61a3-4ce2-b979-b68c935be87f service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:54.787201 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2e02ea-61a3-4ce2-b979-b68c935be87f service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:54.787479 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2e02ea-61a3-4ce2-b979-b68c935be87f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:54.796553 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2e02ea-61a3-4ce2-b979-b68c935be87f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:54.797977 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2e02ea-61a3-4ce2-b979-b68c935be87f service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:54.807978 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1368/2715] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:51:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:51:55.090069 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d4cc4b-e147-4f1c-b7d3-1f529fb5fb35 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:55.091540 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-57d4cc4b-e147-4f1c-b7d3-1f529fb5fb35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:51:55.105751 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d4cc4b-e147-4f1c-b7d3-1f529fb5fb35 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:55.110413 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c67829-3554-4f41-89b8-1d8b1e20e55b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:55.111926 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c67829-3554-4f41-89b8-1d8b1e20e55b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:55.112334 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c67829-3554-4f41-89b8-1d8b1e20e55b admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:55.112512 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c67829-3554-4f41-89b8-1d8b1e20e55b admin admin] PATH_INFO: `/v3/projects/44f228226e624e5589683d0cd27e9374` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:55.114580 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c67829-3554-4f41-89b8-1d8b1e20e55b admin admin] RBAC: Authorizing `identity:delete_project(project_id=44f228226e624e5589683d0cd27e9374)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:55.116748 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c67829-3554-4f41-89b8-1d8b1e20e55b admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:55.165600 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-c2c67829-3554-4f41-89b8-1d8b1e20e55b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44f228226e624e5589683d0cd27e9374', 'request_id': 'req-c2c67829-3554-4f41-89b8-1d8b1e20e55b'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:55.172536 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-c2c67829-3554-4f41-89b8-1d8b1e20e55b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44f228226e624e5589683d0cd27e9374', 'request_id': 'req-c2c67829-3554-4f41-89b8-1d8b1e20e55b'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:55.178556 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-c2c67829-3554-4f41-89b8-1d8b1e20e55b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44f228226e624e5589683d0cd27e9374', 'request_id': 'req-c2c67829-3554-4f41-89b8-1d8b1e20e55b'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:55.189778 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-c2c67829-3554-4f41-89b8-1d8b1e20e55b admin admin] The token cache is being invalidate because project 44f228226e624e5589683d0cd27e9374 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80357) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:51:55.190093 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-c2c67829-3554-4f41-89b8-1d8b1e20e55b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:55.190742 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-c2c67829-3554-4f41-89b8-1d8b1e20e55b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:55.192290 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1348/2716] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jun 16 17:51:55 2022] DELETE /identity/v3/projects/44f228226e624e5589683d0cd27e9374 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:51:55.196335 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2e02ea-61a3-4ce2-b979-b68c935be87f service neutron] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:55.205986 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2e02ea-61a3-4ce2-b979-b68c935be87f service neutron] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:55.261561 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80b3e03-5076-4f76-ab7a-27c556134d47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:55.262700 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80b3e03-5076-4f76-ab7a-27c556134d47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:55.262982 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80b3e03-5076-4f76-ab7a-27c556134d47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:55.263256 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80b3e03-5076-4f76-ab7a-27c556134d47 admin admin] PATH_INFO: `/v3/users/fbd0eae38a3b4545a0d1559731ee6982` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:55.264478 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80b3e03-5076-4f76-ab7a-27c556134d47 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fbd0eae38a3b4545a0d1559731ee6982)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:55.266473 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80b3e03-5076-4f76-ab7a-27c556134d47 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:55.277076 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:51:55.314000 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-b80b3e03-5076-4f76-ab7a-27c556134d47 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fbd0eae38a3b4545a0d1559731ee6982'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:55.325810 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-b80b3e03-5076-4f76-ab7a-27c556134d47 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fbd0eae38a3b4545a0d1559731ee6982'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:55.326483 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-b80b3e03-5076-4f76-ab7a-27c556134d47 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fbd0eae38a3b4545a0d1559731ee6982'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:55.333967 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-b80b3e03-5076-4f76-ab7a-27c556134d47 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fbd0eae38a3b4545a0d1559731ee6982'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:55.341187 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-b80b3e03-5076-4f76-ab7a-27c556134d47 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fbd0eae38a3b4545a0d1559731ee6982'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:55.347094 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1369/2717] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jun 16 17:51:55 2022] DELETE /identity/v3/users/fbd0eae38a3b4545a0d1559731ee6982 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:51:55.635291 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c67829-3554-4f41-89b8-1d8b1e20e55b admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:55.637876 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-c2c67829-3554-4f41-89b8-1d8b1e20e55b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:51:55.664415 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c67829-3554-4f41-89b8-1d8b1e20e55b admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:55.722050 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8484b670-7ccf-4829-9938-b79f4db65751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:55.723577 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8484b670-7ccf-4829-9938-b79f4db65751 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:55.723863 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8484b670-7ccf-4829-9938-b79f4db65751 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:55.724167 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8484b670-7ccf-4829-9938-b79f4db65751 admin admin] PATH_INFO: `/v3/projects/e597a2041fe54a7b8a0719367f668511` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:55.725422 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8484b670-7ccf-4829-9938-b79f4db65751 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e597a2041fe54a7b8a0719367f668511)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:55.727370 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8484b670-7ccf-4829-9938-b79f4db65751 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:55.774473 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-8484b670-7ccf-4829-9938-b79f4db65751 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e597a2041fe54a7b8a0719367f668511', 'request_id': 'req-8484b670-7ccf-4829-9938-b79f4db65751'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:55.781659 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-8484b670-7ccf-4829-9938-b79f4db65751 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e597a2041fe54a7b8a0719367f668511', 'request_id': 'req-8484b670-7ccf-4829-9938-b79f4db65751'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:55.786470 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:51:55.787721 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-8484b670-7ccf-4829-9938-b79f4db65751 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e597a2041fe54a7b8a0719367f668511', 'request_id': 'req-8484b670-7ccf-4829-9938-b79f4db65751'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:55.800218 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-8484b670-7ccf-4829-9938-b79f4db65751 admin admin] The token cache is being invalidate because project e597a2041fe54a7b8a0719367f668511 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80357) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:51:55.800630 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-8484b670-7ccf-4829-9938-b79f4db65751 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:55.801324 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-8484b670-7ccf-4829-9938-b79f4db65751 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:51:55.802877 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1349/2718] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jun 16 17:51:55 2022] DELETE /identity/v3/projects/e597a2041fe54a7b8a0719367f668511 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:51:55.809171 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51edcc5d-dd8b-4419-b70e-b2be46900487 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:55.809655 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51edcc5d-dd8b-4419-b70e-b2be46900487 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:55.809770 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51edcc5d-dd8b-4419-b70e-b2be46900487 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:55.828847 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-51edcc5d-dd8b-4419-b70e-b2be46900487 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:51:55.830047 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-51edcc5d-dd8b-4419-b70e-b2be46900487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:51:55.899018 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1370/2719] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:51:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:51:55.906952 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b1f567-1dea-4a1c-816d-830cc6cda8e2 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:55.907239 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b1f567-1dea-4a1c-816d-830cc6cda8e2 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:55.907514 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b1f567-1dea-4a1c-816d-830cc6cda8e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:55.926334 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-b1b1f567-1dea-4a1c-816d-830cc6cda8e2 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:51:55.927524 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-b1b1f567-1dea-4a1c-816d-830cc6cda8e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:51:55.935248 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1350/2720] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:51:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:51:55.939324 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51edcc5d-dd8b-4419-b70e-b2be46900487 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:55.950344 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51edcc5d-dd8b-4419-b70e-b2be46900487 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:55.955145 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0157fd79-edcc-4ec5-b313-d2bfe299e595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:55.955802 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0157fd79-edcc-4ec5-b313-d2bfe299e595 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:55.956091 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0157fd79-edcc-4ec5-b313-d2bfe299e595 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:55.956393 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0157fd79-edcc-4ec5-b313-d2bfe299e595 admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:55.957628 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0157fd79-edcc-4ec5-b313-d2bfe299e595 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:51:55.958024 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0157fd79-edcc-4ec5-b313-d2bfe299e595 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:55.959666 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0157fd79-edcc-4ec5-b313-d2bfe299e595 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:55.971371 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1371/2721] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jun 16 17:51:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:51:55.977422 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b1f567-1dea-4a1c-816d-830cc6cda8e2 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:55.986624 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b1f567-1dea-4a1c-816d-830cc6cda8e2 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:55.992436 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412f80dc-7a7d-4485-be4f-a38af26f8f89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:55.993107 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412f80dc-7a7d-4485-be4f-a38af26f8f89 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:55.993374 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412f80dc-7a7d-4485-be4f-a38af26f8f89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:55.993649 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412f80dc-7a7d-4485-be4f-a38af26f8f89 admin admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:55.994346 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412f80dc-7a7d-4485-be4f-a38af26f8f89 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:55.996248 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412f80dc-7a7d-4485-be4f-a38af26f8f89 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:56.027366 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1351/2722] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jun 16 17:51:55 2022] POST /identity/v3/projects => generated 377 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:51:56.028984 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0157fd79-edcc-4ec5-b313-d2bfe299e595 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:56.039899 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0157fd79-edcc-4ec5-b313-d2bfe299e595 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:56.100302 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db90e057-5d30-44dc-892d-f5fbac6aead3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:56.101033 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db90e057-5d30-44dc-892d-f5fbac6aead3 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:56.101323 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db90e057-5d30-44dc-892d-f5fbac6aead3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:56.101606 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db90e057-5d30-44dc-892d-f5fbac6aead3 admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:56.102292 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db90e057-5d30-44dc-892d-f5fbac6aead3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:56.104210 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db90e057-5d30-44dc-892d-f5fbac6aead3 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:56.123473 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1372/2723] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jun 16 17:51:56 2022] POST /identity/v3/users => generated 289 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:51:56.127573 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412f80dc-7a7d-4485-be4f-a38af26f8f89 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:56.138404 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412f80dc-7a7d-4485-be4f-a38af26f8f89 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:56.143243 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f79d4c1-6a7c-4c22-8fe7-1bc5f5c40214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:56.143903 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f79d4c1-6a7c-4c22-8fe7-1bc5f5c40214 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:56.144783 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f79d4c1-6a7c-4c22-8fe7-1bc5f5c40214 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:56.145200 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f79d4c1-6a7c-4c22-8fe7-1bc5f5c40214 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:56.145932 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f79d4c1-6a7c-4c22-8fe7-1bc5f5c40214 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:56.147514 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f79d4c1-6a7c-4c22-8fe7-1bc5f5c40214 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:56.157822 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1352/2724] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:51:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:51:56.162362 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db90e057-5d30-44dc-892d-f5fbac6aead3 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:56.168677 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:51:56.172678 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db90e057-5d30-44dc-892d-f5fbac6aead3 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:56.179188 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ddda81-8e65-41b5-a6be-fc819c9688c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:56.180039 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ddda81-8e65-41b5-a6be-fc819c9688c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:56.180529 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ddda81-8e65-41b5-a6be-fc819c9688c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:56.180853 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ddda81-8e65-41b5-a6be-fc819c9688c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:56.181785 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ddda81-8e65-41b5-a6be-fc819c9688c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:56.183405 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ddda81-8e65-41b5-a6be-fc819c9688c7 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:56.195669 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1373/2725] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:51:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:51:56.199504 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f79d4c1-6a7c-4c22-8fe7-1bc5f5c40214 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:56.210329 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f79d4c1-6a7c-4c22-8fe7-1bc5f5c40214 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:56.215606 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a42b71-2529-4922-9500-785b835ec418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:56.216579 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a42b71-2529-4922-9500-785b835ec418 admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:56.216873 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a42b71-2529-4922-9500-785b835ec418 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:56.217156 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a42b71-2529-4922-9500-785b835ec418 admin admin] PATH_INFO: `/v3/projects/9c249c315494483892639b1213d4bc5d/users/31e04db1b0f6444e9a884a0e58ba38da/roles/0be2075946f54c8485f1c275f746478a` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:56.232899 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a42b71-2529-4922-9500-785b835ec418 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c249c315494483892639b1213d4bc5d, user_id=31e04db1b0f6444e9a884a0e58ba38da, role_id=0be2075946f54c8485f1c275f746478a)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:56.235608 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a42b71-2529-4922-9500-785b835ec418 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:56.248846 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1353/2726] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:51:56 2022] PUT /identity/v3/projects/9c249c315494483892639b1213d4bc5d/users/31e04db1b0f6444e9a884a0e58ba38da/roles/0be2075946f54c8485f1c275f746478a => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:51:56.254564 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb70e35-7d2b-4d59-a056-7a2fe1372e09 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:56.254865 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb70e35-7d2b-4d59-a056-7a2fe1372e09 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:56.255152 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb70e35-7d2b-4d59-a056-7a2fe1372e09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:56.272885 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-fbb70e35-7d2b-4d59-a056-7a2fe1372e09 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:51:56.274117 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-fbb70e35-7d2b-4d59-a056-7a2fe1372e09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:51:56.334914 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1374/2727] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:51:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:51:56.344464 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a42b71-2529-4922-9500-785b835ec418 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:56.346175 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-16a42b71-2529-4922-9500-785b835ec418 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:51:56.404522 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a42b71-2529-4922-9500-785b835ec418 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:56.408067 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb32cd5-39eb-4a23-8e8d-16429a1bcc24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:56.408735 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb32cd5-39eb-4a23-8e8d-16429a1bcc24 service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:56.409016 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb32cd5-39eb-4a23-8e8d-16429a1bcc24 service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:56.409292 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb32cd5-39eb-4a23-8e8d-16429a1bcc24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:56.418882 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb32cd5-39eb-4a23-8e8d-16429a1bcc24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:56.420308 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb32cd5-39eb-4a23-8e8d-16429a1bcc24 service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:56.430206 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1354/2728] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:51:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:51:58.988489 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ef21d0-105b-4806-ae3b-89d19f3ab001 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:58.989675 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ef21d0-105b-4806-ae3b-89d19f3ab001 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:58.989675 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ef21d0-105b-4806-ae3b-89d19f3ab001 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:59.028881 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-79ef21d0-105b-4806-ae3b-89d19f3ab001 None None] MFA Rules not processed for user `71443c30b4554719b5549493bf3cc00c`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:51:59.032188 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-79ef21d0-105b-4806-ae3b-89d19f3ab001 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:51:59.156495 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1375/2729] 10.222.0.42 () {58 vars in 1009 bytes} [Thu Jun 16 17:51:58 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:51:59.234130 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb32cd5-39eb-4a23-8e8d-16429a1bcc24 service neutron] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:59.235609 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-fcb32cd5-39eb-4a23-8e8d-16429a1bcc24 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:51:59.391551 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb32cd5-39eb-4a23-8e8d-16429a1bcc24 service neutron] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:59.400263 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600b8800-a134-4d93-bbf9-8bca87bc1854 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47b7a9cd615c41d6983a0007b475038c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:59.401965 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600b8800-a134-4d93-bbf9-8bca87bc1854 service swift] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:59.401965 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600b8800-a134-4d93-bbf9-8bca87bc1854 service swift] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:59.404607 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600b8800-a134-4d93-bbf9-8bca87bc1854 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:59.417295 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600b8800-a134-4d93-bbf9-8bca87bc1854 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:59.418904 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600b8800-a134-4d93-bbf9-8bca87bc1854 service swift] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:59.429232 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1355/2730] 10.222.0.42 () {60 vars in 1384 bytes} [Thu Jun 16 17:51:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 17:52:01.379948 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c95fbce-59ca-43b6-964c-a00f4e1172f7 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:52:01.384877 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c95fbce-59ca-43b6-964c-a00f4e1172f7 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:52:01.385295 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c95fbce-59ca-43b6-964c-a00f4e1172f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:52:01.423748 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-4c95fbce-59ca-43b6-964c-a00f4e1172f7 None None] MFA Rules not processed for user `31e04db1b0f6444e9a884a0e58ba38da`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:52:01.425994 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-4c95fbce-59ca-43b6-964c-a00f4e1172f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:52:01.521432 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1376/2731] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:52:01 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:52:01.526392 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600b8800-a134-4d93-bbf9-8bca87bc1854 service swift] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:52:01.544858 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600b8800-a134-4d93-bbf9-8bca87bc1854 service swift] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:52:01.570713 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ea0dca-98e2-43ce-8732-5b17f007a590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:52:01.571424 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ea0dca-98e2-43ce-8732-5b17f007a590 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:52:01.572034 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ea0dca-98e2-43ce-8732-5b17f007a590 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:52:01.572308 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ea0dca-98e2-43ce-8732-5b17f007a590 admin admin] PATH_INFO: `/v3/projects` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:52:01.573196 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ea0dca-98e2-43ce-8732-5b17f007a590 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:52:01.575408 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ea0dca-98e2-43ce-8732-5b17f007a590 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:52:01.609997 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:52:01.626651 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1356/2732] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jun 16 17:52:01 2022] POST /identity/v3/projects => generated 377 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:52:01.631422 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c95fbce-59ca-43b6-964c-a00f4e1172f7 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:52:01.648977 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c95fbce-59ca-43b6-964c-a00f4e1172f7 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:52:01.692931 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:52:01.768328 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d64a8a-596d-49d9-8cdd-e4169ec1d694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:52:01.769754 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d64a8a-596d-49d9-8cdd-e4169ec1d694 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:52:01.770363 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d64a8a-596d-49d9-8cdd-e4169ec1d694 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:52:01.770848 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d64a8a-596d-49d9-8cdd-e4169ec1d694 admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:52:01.771710 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d64a8a-596d-49d9-8cdd-e4169ec1d694 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:52:01.775240 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d64a8a-596d-49d9-8cdd-e4169ec1d694 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:52:01.822185 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1377/2733] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jun 16 17:52:01 2022] POST /identity/v3/users => generated 295 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:52:01.823681 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ea0dca-98e2-43ce-8732-5b17f007a590 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:52:01.838695 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ea0dca-98e2-43ce-8732-5b17f007a590 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:52:01.846481 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a8f995-2806-452e-bab4-0e9a5d059336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:52:01.847906 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a8f995-2806-452e-bab4-0e9a5d059336 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:52:01.848540 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a8f995-2806-452e-bab4-0e9a5d059336 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:52:01.849330 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a8f995-2806-452e-bab4-0e9a5d059336 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:52:01.850194 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a8f995-2806-452e-bab4-0e9a5d059336 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:52:01.852803 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a8f995-2806-452e-bab4-0e9a5d059336 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:52:01.876418 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1357/2734] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:52:01 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:52:01.880099 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d64a8a-596d-49d9-8cdd-e4169ec1d694 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:52:01.893461 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d64a8a-596d-49d9-8cdd-e4169ec1d694 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:52:01.899265 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f647425-e2cd-4ffa-9dd0-a87cd7582eac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:52:01.900575 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f647425-e2cd-4ffa-9dd0-a87cd7582eac admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:52:01.900962 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f647425-e2cd-4ffa-9dd0-a87cd7582eac admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:52:01.901184 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f647425-e2cd-4ffa-9dd0-a87cd7582eac admin admin] PATH_INFO: `/v3/projects/d7254274d5fd4e4c88d98ebc4ad9b47a/users/a6da6754c62c4da6bb4a264456d2cf01/roles/3ca2dcbbd29d4a31a3ab78f2299bc239` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:52:01.918773 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f647425-e2cd-4ffa-9dd0-a87cd7582eac admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7254274d5fd4e4c88d98ebc4ad9b47a, user_id=a6da6754c62c4da6bb4a264456d2cf01, role_id=3ca2dcbbd29d4a31a3ab78f2299bc239)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:52:01.921995 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f647425-e2cd-4ffa-9dd0-a87cd7582eac admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:52:01.941976 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1378/2735] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:52:01 2022] PUT /identity/v3/projects/d7254274d5fd4e4c88d98ebc4ad9b47a/users/a6da6754c62c4da6bb4a264456d2cf01/roles/3ca2dcbbd29d4a31a3ab78f2299bc239 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:52:01.943333 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a8f995-2806-452e-bab4-0e9a5d059336 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:52:01.955363 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a8f995-2806-452e-bab4-0e9a5d059336 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:52:02.034376 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf51273-5e76-4a91-83cc-7d85f8070075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:52:02.036837 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf51273-5e76-4a91-83cc-7d85f8070075 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:52:02.037752 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf51273-5e76-4a91-83cc-7d85f8070075 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:52:02.038212 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf51273-5e76-4a91-83cc-7d85f8070075 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:52:02.039030 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf51273-5e76-4a91-83cc-7d85f8070075 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:52:02.040811 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf51273-5e76-4a91-83cc-7d85f8070075 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:52:02.076732 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f647425-e2cd-4ffa-9dd0-a87cd7582eac admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:52:02.084739 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1358/2736] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:52:01 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:52:02.090313 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f647425-e2cd-4ffa-9dd0-a87cd7582eac admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:52:02.095827 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f478a2-3efe-49f6-9f46-b2726278a20f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:52:02.099187 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f478a2-3efe-49f6-9f46-b2726278a20f admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:52:02.099550 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f478a2-3efe-49f6-9f46-b2726278a20f admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:52:02.099827 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f478a2-3efe-49f6-9f46-b2726278a20f admin admin] PATH_INFO: `/v3/projects/d7254274d5fd4e4c88d98ebc4ad9b47a/users/a6da6754c62c4da6bb4a264456d2cf01/roles/3ca2dcbbd29d4a31a3ab78f2299bc239` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:52:02.102451 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f478a2-3efe-49f6-9f46-b2726278a20f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7254274d5fd4e4c88d98ebc4ad9b47a, user_id=a6da6754c62c4da6bb4a264456d2cf01, role_id=3ca2dcbbd29d4a31a3ab78f2299bc239)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:52:02.105497 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f478a2-3efe-49f6-9f46-b2726278a20f admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:52:02.123163 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1379/2737] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:52:02 2022] PUT /identity/v3/projects/d7254274d5fd4e4c88d98ebc4ad9b47a/users/a6da6754c62c4da6bb4a264456d2cf01/roles/3ca2dcbbd29d4a31a3ab78f2299bc239 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:52:02.647371 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf51273-5e76-4a91-83cc-7d85f8070075 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:52:02.651634 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-0cf51273-5e76-4a91-83cc-7d85f8070075 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:52:02.675016 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf51273-5e76-4a91-83cc-7d85f8070075 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:52:02.819257 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050dd919-c65a-4ac8-aac9-78e8db222523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:52:02.821317 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050dd919-c65a-4ac8-aac9-78e8db222523 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:52:02.821615 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050dd919-c65a-4ac8-aac9-78e8db222523 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:52:02.821899 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050dd919-c65a-4ac8-aac9-78e8db222523 admin admin] PATH_INFO: `/v3/users/0723640a0162478dbff3f194f4923362` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:52:02.823278 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050dd919-c65a-4ac8-aac9-78e8db222523 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0723640a0162478dbff3f194f4923362)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:52:02.828776 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050dd919-c65a-4ac8-aac9-78e8db222523 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:52:02.934201 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-050dd919-c65a-4ac8-aac9-78e8db222523 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0723640a0162478dbff3f194f4923362'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:52:02.936433 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-050dd919-c65a-4ac8-aac9-78e8db222523 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0723640a0162478dbff3f194f4923362'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:52:02.955923 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-050dd919-c65a-4ac8-aac9-78e8db222523 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0723640a0162478dbff3f194f4923362'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:52:02.958711 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-050dd919-c65a-4ac8-aac9-78e8db222523 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0723640a0162478dbff3f194f4923362'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:52:02.978638 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-050dd919-c65a-4ac8-aac9-78e8db222523 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0723640a0162478dbff3f194f4923362'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:52:02.996453 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1359/2738] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jun 16 17:52:02 2022] DELETE /identity/v3/users/0723640a0162478dbff3f194f4923362 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:52:03.000525 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ede544-4a1e-4315-bb72-8eda750ad240 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:52:03.001326 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ede544-4a1e-4315-bb72-8eda750ad240 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:52:03.001622 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ede544-4a1e-4315-bb72-8eda750ad240 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:52:03.027845 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-c7ede544-4a1e-4315-bb72-8eda750ad240 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:52:03.029121 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-c7ede544-4a1e-4315-bb72-8eda750ad240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:52:03.169443 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1380/2739] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:52:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:52:03.181347 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050dd919-c65a-4ac8-aac9-78e8db222523 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:52:03.183369 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-050dd919-c65a-4ac8-aac9-78e8db222523 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:52:03.192268 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:52:03.324479 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050dd919-c65a-4ac8-aac9-78e8db222523 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:52:03.331223 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff49ad1-ae2e-47c0-bc57-29322c5f3ef1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:52:03.333993 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff49ad1-ae2e-47c0-bc57-29322c5f3ef1 service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:52:03.334815 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff49ad1-ae2e-47c0-bc57-29322c5f3ef1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:52:03.335529 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff49ad1-ae2e-47c0-bc57-29322c5f3ef1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:52:03.356200 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff49ad1-ae2e-47c0-bc57-29322c5f3ef1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:52:03.358372 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff49ad1-ae2e-47c0-bc57-29322c5f3ef1 service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:52:03.376767 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1360/2740] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:52:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:52:04.510431 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ede544-4a1e-4315-bb72-8eda750ad240 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:52:04.512736 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-c7ede544-4a1e-4315-bb72-8eda750ad240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:52:04.528300 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ede544-4a1e-4315-bb72-8eda750ad240 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:52:04.538057 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dd0cb4-537b-4229-ab60-7012597f3199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:52:04.539188 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dd0cb4-537b-4229-ab60-7012597f3199 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:52:04.541038 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dd0cb4-537b-4229-ab60-7012597f3199 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:52:04.541735 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dd0cb4-537b-4229-ab60-7012597f3199 admin admin] PATH_INFO: `/v3/projects/e6a2a4d7477841f0879ab8c2be04f7b2` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:52:04.543885 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dd0cb4-537b-4229-ab60-7012597f3199 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6a2a4d7477841f0879ab8c2be04f7b2)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:52:04.546452 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dd0cb4-537b-4229-ab60-7012597f3199 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:52:04.643668 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-76dd0cb4-537b-4229-ab60-7012597f3199 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6a2a4d7477841f0879ab8c2be04f7b2', 'request_id': 'req-76dd0cb4-537b-4229-ab60-7012597f3199'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:52:04.662379 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-76dd0cb4-537b-4229-ab60-7012597f3199 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6a2a4d7477841f0879ab8c2be04f7b2', 'request_id': 'req-76dd0cb4-537b-4229-ab60-7012597f3199'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:52:04.692846 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-76dd0cb4-537b-4229-ab60-7012597f3199 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6a2a4d7477841f0879ab8c2be04f7b2', 'request_id': 'req-76dd0cb4-537b-4229-ab60-7012597f3199'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:52:04.714592 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-76dd0cb4-537b-4229-ab60-7012597f3199 admin admin] The token cache is being invalidate because project e6a2a4d7477841f0879ab8c2be04f7b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:52:04.714907 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-76dd0cb4-537b-4229-ab60-7012597f3199 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:52:04.715778 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-76dd0cb4-537b-4229-ab60-7012597f3199 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:52:04.718662 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1381/2741] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jun 16 17:52:04 2022] DELETE /identity/v3/projects/e6a2a4d7477841f0879ab8c2be04f7b2 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:52:04.722939 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff49ad1-ae2e-47c0-bc57-29322c5f3ef1 service neutron] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:52:04.740332 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff49ad1-ae2e-47c0-bc57-29322c5f3ef1 service neutron] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:52:04.879930 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a9f326-34d5-4f3e-8932-2a98c93f8a7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:52:04.885907 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a9f326-34d5-4f3e-8932-2a98c93f8a7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:52:04.886961 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a9f326-34d5-4f3e-8932-2a98c93f8a7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:52:04.889180 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a9f326-34d5-4f3e-8932-2a98c93f8a7a admin admin] PATH_INFO: `/v3/users/d3599cf0162f479d8a138bdc618313c7` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:52:04.898204 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a9f326-34d5-4f3e-8932-2a98c93f8a7a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3599cf0162f479d8a138bdc618313c7)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:52:04.900684 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a9f326-34d5-4f3e-8932-2a98c93f8a7a admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:52:04.979053 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-75a9f326-34d5-4f3e-8932-2a98c93f8a7a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3599cf0162f479d8a138bdc618313c7'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:52:04.980404 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-75a9f326-34d5-4f3e-8932-2a98c93f8a7a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3599cf0162f479d8a138bdc618313c7'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:52:05.000661 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-75a9f326-34d5-4f3e-8932-2a98c93f8a7a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3599cf0162f479d8a138bdc618313c7'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:52:05.002104 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-75a9f326-34d5-4f3e-8932-2a98c93f8a7a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3599cf0162f479d8a138bdc618313c7'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:52:05.029766 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-75a9f326-34d5-4f3e-8932-2a98c93f8a7a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3599cf0162f479d8a138bdc618313c7'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:52:05.059653 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1361/2742] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jun 16 17:52:04 2022] DELETE /identity/v3/users/d3599cf0162f479d8a138bdc618313c7 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:52:05.926568 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dd0cb4-537b-4229-ab60-7012597f3199 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:52:05.929362 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-76dd0cb4-537b-4229-ab60-7012597f3199 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:52:05.949010 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:52:05.954955 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dd0cb4-537b-4229-ab60-7012597f3199 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:52:06.119602 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5195c10e-2301-4e5f-af7b-aff6750ee3fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:52:06.120552 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5195c10e-2301-4e5f-af7b-aff6750ee3fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:52:06.120814 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5195c10e-2301-4e5f-af7b-aff6750ee3fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:52:06.121094 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5195c10e-2301-4e5f-af7b-aff6750ee3fc admin admin] PATH_INFO: `/v3/projects/8f7569d32caf4229ae825a21633bbc58` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:52:06.127453 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5195c10e-2301-4e5f-af7b-aff6750ee3fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f7569d32caf4229ae825a21633bbc58)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:52:06.130085 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5195c10e-2301-4e5f-af7b-aff6750ee3fc admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:52:06.373143 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-5195c10e-2301-4e5f-af7b-aff6750ee3fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f7569d32caf4229ae825a21633bbc58', 'request_id': 'req-5195c10e-2301-4e5f-af7b-aff6750ee3fc'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:52:06.395387 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-5195c10e-2301-4e5f-af7b-aff6750ee3fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f7569d32caf4229ae825a21633bbc58', 'request_id': 'req-5195c10e-2301-4e5f-af7b-aff6750ee3fc'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:52:06.411347 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-5195c10e-2301-4e5f-af7b-aff6750ee3fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f7569d32caf4229ae825a21633bbc58', 'request_id': 'req-5195c10e-2301-4e5f-af7b-aff6750ee3fc'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:52:06.454840 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-5195c10e-2301-4e5f-af7b-aff6750ee3fc admin admin] The token cache is being invalidate because project 8f7569d32caf4229ae825a21633bbc58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:52:06.455135 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-5195c10e-2301-4e5f-af7b-aff6750ee3fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:52:06.455820 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-5195c10e-2301-4e5f-af7b-aff6750ee3fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:52:06.466950 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1382/2743] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jun 16 17:52:05 2022] DELETE /identity/v3/projects/8f7569d32caf4229ae825a21633bbc58 => generated 0 bytes in 541 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:52:06.471918 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7935cc-8279-4f3a-b3d9-0eadf5ce72e3 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:52:06.473128 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7935cc-8279-4f3a-b3d9-0eadf5ce72e3 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:52:06.473470 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7935cc-8279-4f3a-b3d9-0eadf5ce72e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:52:06.528305 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-7b7935cc-8279-4f3a-b3d9-0eadf5ce72e3 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:52:06.532493 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-7b7935cc-8279-4f3a-b3d9-0eadf5ce72e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:52:06.608537 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:52:06.744941 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1362/2744] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:52:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:52:06.748629 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc2192a-e665-401a-bf26-db3d6672a922 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:52:06.749123 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc2192a-e665-401a-bf26-db3d6672a922 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:52:06.749558 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc2192a-e665-401a-bf26-db3d6672a922 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:52:06.812927 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-6dc2192a-e665-401a-bf26-db3d6672a922 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:52:06.814470 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-6dc2192a-e665-401a-bf26-db3d6672a922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:52:06.831388 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1383/2745] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:52:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:52:06.836916 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7935cc-8279-4f3a-b3d9-0eadf5ce72e3 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:52:06.855788 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7935cc-8279-4f3a-b3d9-0eadf5ce72e3 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:52:06.885453 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4b7405-45f9-4f89-9f40-9b9523eb8a87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:52:06.888468 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4b7405-45f9-4f89-9f40-9b9523eb8a87 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:52:06.889049 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4b7405-45f9-4f89-9f40-9b9523eb8a87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:52:06.891860 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4b7405-45f9-4f89-9f40-9b9523eb8a87 admin admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:52:06.894908 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4b7405-45f9-4f89-9f40-9b9523eb8a87 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:52:06.903963 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4b7405-45f9-4f89-9f40-9b9523eb8a87 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:52:06.909785 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4b7405-45f9-4f89-9f40-9b9523eb8a87 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:52:06.988616 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1363/2746] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jun 16 17:52:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:52:06.989778 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc2192a-e665-401a-bf26-db3d6672a922 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:52:07.012332 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc2192a-e665-401a-bf26-db3d6672a922 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:52:07.029918 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97af361-f322-48b7-8ae1-62a03b5b2e1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:52:07.030855 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97af361-f322-48b7-8ae1-62a03b5b2e1a admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:52:07.031326 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97af361-f322-48b7-8ae1-62a03b5b2e1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:52:07.031755 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97af361-f322-48b7-8ae1-62a03b5b2e1a admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:52:07.032826 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97af361-f322-48b7-8ae1-62a03b5b2e1a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:52:07.035552 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97af361-f322-48b7-8ae1-62a03b5b2e1a admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:52:07.137917 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1384/2747] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jun 16 17:52:06 2022] POST /identity/v3/projects => generated 369 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:52:07.142075 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4b7405-45f9-4f89-9f40-9b9523eb8a87 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:52:07.163188 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4b7405-45f9-4f89-9f40-9b9523eb8a87 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:52:07.267882 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e573970b-ba00-4a1e-bbb2-395e9dd874f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:52:07.268663 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e573970b-ba00-4a1e-bbb2-395e9dd874f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:52:07.269694 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e573970b-ba00-4a1e-bbb2-395e9dd874f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:52:07.269869 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e573970b-ba00-4a1e-bbb2-395e9dd874f9 admin admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:52:07.270667 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e573970b-ba00-4a1e-bbb2-395e9dd874f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:52:07.272463 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e573970b-ba00-4a1e-bbb2-395e9dd874f9 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:52:07.311851 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1364/2748] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jun 16 17:52:07 2022] POST /identity/v3/users => generated 285 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:52:07.313131 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97af361-f322-48b7-8ae1-62a03b5b2e1a admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:52:07.326911 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97af361-f322-48b7-8ae1-62a03b5b2e1a admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:52:07.333987 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7cc269-bb6c-42af-9f4e-a9999e7fcdea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:52:07.340840 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7cc269-bb6c-42af-9f4e-a9999e7fcdea admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:52:07.341614 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7cc269-bb6c-42af-9f4e-a9999e7fcdea admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:52:07.342155 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7cc269-bb6c-42af-9f4e-a9999e7fcdea admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:52:07.343470 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7cc269-bb6c-42af-9f4e-a9999e7fcdea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:52:07.345714 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7cc269-bb6c-42af-9f4e-a9999e7fcdea admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:52:07.361321 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1385/2749] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:52:07 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:52:07.366059 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e573970b-ba00-4a1e-bbb2-395e9dd874f9 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:52:07.381773 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e573970b-ba00-4a1e-bbb2-395e9dd874f9 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:52:07.389743 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172bd489-cd4e-4a56-8eaa-a0e2cbd73063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:52:07.390437 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172bd489-cd4e-4a56-8eaa-a0e2cbd73063 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:52:07.390725 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172bd489-cd4e-4a56-8eaa-a0e2cbd73063 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:52:07.390994 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172bd489-cd4e-4a56-8eaa-a0e2cbd73063 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:52:07.391727 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172bd489-cd4e-4a56-8eaa-a0e2cbd73063 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:52:07.396851 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172bd489-cd4e-4a56-8eaa-a0e2cbd73063 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:52:07.414776 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1365/2750] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:52:07 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:52:07.419402 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7cc269-bb6c-42af-9f4e-a9999e7fcdea admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:52:07.442894 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7cc269-bb6c-42af-9f4e-a9999e7fcdea admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:52:07.469515 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ec7681-0540-49bd-9c4a-e991978b3db8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:52:07.479821 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ec7681-0540-49bd-9c4a-e991978b3db8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:52:07.482081 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ec7681-0540-49bd-9c4a-e991978b3db8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:52:07.482744 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ec7681-0540-49bd-9c4a-e991978b3db8 admin admin] PATH_INFO: `/v3/projects/873d76fb7094436ea3c5123aa959a0da/users/d47e4a43c01d447f952544072c560e73/roles/0be2075946f54c8485f1c275f746478a` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:52:07.551476 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ec7681-0540-49bd-9c4a-e991978b3db8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=873d76fb7094436ea3c5123aa959a0da, user_id=d47e4a43c01d447f952544072c560e73, role_id=0be2075946f54c8485f1c275f746478a)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:52:07.563235 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ec7681-0540-49bd-9c4a-e991978b3db8 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:52:07.622427 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1386/2751] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:52:07 2022] PUT /identity/v3/projects/873d76fb7094436ea3c5123aa959a0da/users/d47e4a43c01d447f952544072c560e73/roles/0be2075946f54c8485f1c275f746478a => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:52:07.642121 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db17b2d4-db59-4624-832f-abad14611227 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:52:07.643137 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db17b2d4-db59-4624-832f-abad14611227 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:52:07.643582 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db17b2d4-db59-4624-832f-abad14611227 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:52:07.729895 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-db17b2d4-db59-4624-832f-abad14611227 None None] MFA Rules not processed for user `d47e4a43c01d447f952544072c560e73`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:52:07.748803 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-db17b2d4-db59-4624-832f-abad14611227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:52:07.997893 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ec7681-0540-49bd-9c4a-e991978b3db8 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:52:08.000832 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1366/2752] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:52:07 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 360 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:52:08.177734 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:52:08.318332 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ec7681-0540-49bd-9c4a-e991978b3db8 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:52:08.365963 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21650896-f2c2-4bbb-ba9b-0ddd700f19af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:52:08.366657 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21650896-f2c2-4bbb-ba9b-0ddd700f19af admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:52:08.367148 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21650896-f2c2-4bbb-ba9b-0ddd700f19af admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:52:08.368129 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21650896-f2c2-4bbb-ba9b-0ddd700f19af admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:52:08.369036 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21650896-f2c2-4bbb-ba9b-0ddd700f19af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:52:08.383361 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21650896-f2c2-4bbb-ba9b-0ddd700f19af admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:52:08.477061 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1387/2753] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jun 16 17:52:07 2022] POST /identity/v3/projects => generated 367 bytes in 480 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:52:08.486246 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db17b2d4-db59-4624-832f-abad14611227 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:52:08.501465 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db17b2d4-db59-4624-832f-abad14611227 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:52:08.528732 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:52:08.594427 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e464953c-e5f0-4008-9c92-4f4688d63612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:52:08.595051 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e464953c-e5f0-4008-9c92-4f4688d63612 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:52:08.595335 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e464953c-e5f0-4008-9c92-4f4688d63612 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:52:08.595611 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e464953c-e5f0-4008-9c92-4f4688d63612 admin admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:52:08.596308 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e464953c-e5f0-4008-9c92-4f4688d63612 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:52:08.600113 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e464953c-e5f0-4008-9c92-4f4688d63612 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:52:08.645471 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21650896-f2c2-4bbb-ba9b-0ddd700f19af admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:52:08.646346 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1367/2754] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jun 16 17:52:08 2022] POST /identity/v3/users => generated 290 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:52:08.663901 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21650896-f2c2-4bbb-ba9b-0ddd700f19af admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:52:08.672552 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a513b7c4-16c7-482d-a1bf-d99bb86b9d5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:52:08.673502 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a513b7c4-16c7-482d-a1bf-d99bb86b9d5f admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:52:08.673954 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a513b7c4-16c7-482d-a1bf-d99bb86b9d5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:52:08.674408 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a513b7c4-16c7-482d-a1bf-d99bb86b9d5f admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:52:08.675835 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a513b7c4-16c7-482d-a1bf-d99bb86b9d5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:52:08.678481 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a513b7c4-16c7-482d-a1bf-d99bb86b9d5f admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:52:08.704108 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e464953c-e5f0-4008-9c92-4f4688d63612 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:52:08.717170 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1388/2755] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:52:08 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:52:08.719626 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e464953c-e5f0-4008-9c92-4f4688d63612 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:52:08.729604 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edac4db-f98c-4fe0-98fc-537025d8bbb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:52:08.731392 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edac4db-f98c-4fe0-98fc-537025d8bbb7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:52:08.731945 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edac4db-f98c-4fe0-98fc-537025d8bbb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:52:08.732592 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edac4db-f98c-4fe0-98fc-537025d8bbb7 admin admin] PATH_INFO: `/v3/projects/c1ad546c304c494d8d7e455456817394/users/a9149060f18246378e9fa928d055eb2a/roles/3ca2dcbbd29d4a31a3ab78f2299bc239` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:52:08.756537 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edac4db-f98c-4fe0-98fc-537025d8bbb7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1ad546c304c494d8d7e455456817394, user_id=a9149060f18246378e9fa928d055eb2a, role_id=3ca2dcbbd29d4a31a3ab78f2299bc239)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:52:08.759675 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edac4db-f98c-4fe0-98fc-537025d8bbb7 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:52:08.791404 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1368/2756] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:52:08 2022] PUT /identity/v3/projects/c1ad546c304c494d8d7e455456817394/users/a9149060f18246378e9fa928d055eb2a/roles/3ca2dcbbd29d4a31a3ab78f2299bc239 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:52:08.792855 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a513b7c4-16c7-482d-a1bf-d99bb86b9d5f admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:52:08.810851 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a513b7c4-16c7-482d-a1bf-d99bb86b9d5f admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:52:09.001021 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae28faf-d148-4c60-844c-bed59df2d505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:52:09.001716 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae28faf-d148-4c60-844c-bed59df2d505 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:52:09.001988 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae28faf-d148-4c60-844c-bed59df2d505 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:52:09.002269 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae28faf-d148-4c60-844c-bed59df2d505 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:52:09.003142 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae28faf-d148-4c60-844c-bed59df2d505 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:52:09.006429 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae28faf-d148-4c60-844c-bed59df2d505 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:52:09.031784 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1389/2757] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:52:08 2022] GET /identity/v3/roles => generated 1420 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:52:09.033313 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edac4db-f98c-4fe0-98fc-537025d8bbb7 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:52:09.047660 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edac4db-f98c-4fe0-98fc-537025d8bbb7 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:52:09.062490 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e279b354-6707-4ed1-8c99-39fb4cb67188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:52:09.063686 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e279b354-6707-4ed1-8c99-39fb4cb67188 admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:52:09.063985 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e279b354-6707-4ed1-8c99-39fb4cb67188 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:52:09.065387 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e279b354-6707-4ed1-8c99-39fb4cb67188 admin admin] PATH_INFO: `/v3/projects/c1ad546c304c494d8d7e455456817394/users/a9149060f18246378e9fa928d055eb2a/roles/3ca2dcbbd29d4a31a3ab78f2299bc239` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:52:09.072580 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e279b354-6707-4ed1-8c99-39fb4cb67188 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1ad546c304c494d8d7e455456817394, user_id=a9149060f18246378e9fa928d055eb2a, role_id=3ca2dcbbd29d4a31a3ab78f2299bc239)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:52:09.079530 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e279b354-6707-4ed1-8c99-39fb4cb67188 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:52:09.117487 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1369/2758] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:52:09 2022] PUT /identity/v3/projects/c1ad546c304c494d8d7e455456817394/users/a9149060f18246378e9fa928d055eb2a/roles/3ca2dcbbd29d4a31a3ab78f2299bc239 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:52:09.122598 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f030176-6141-4f06-988b-a184bcecf81d None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:52:09.123226 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f030176-6141-4f06-988b-a184bcecf81d None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:52:09.124043 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f030176-6141-4f06-988b-a184bcecf81d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:52:09.175259 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-2f030176-6141-4f06-988b-a184bcecf81d None None] MFA Rules not processed for user `a9149060f18246378e9fa928d055eb2a`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:52:09.177024 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-2f030176-6141-4f06-988b-a184bcecf81d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:52:09.352420 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1390/2759] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:52:09 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:52:09.365943 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e279b354-6707-4ed1-8c99-39fb4cb67188 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:52:09.368756 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-e279b354-6707-4ed1-8c99-39fb4cb67188 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:52:09.510700 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e279b354-6707-4ed1-8c99-39fb4cb67188 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:52:09.511147 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a49d9e-cce2-494a-a4e9-6fbc9e59d35d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24d5e2637b7043e08313909d5b06e329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:52:09.511943 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a49d9e-cce2-494a-a4e9-6fbc9e59d35d service cinder] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:52:09.512619 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a49d9e-cce2-494a-a4e9-6fbc9e59d35d service cinder] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:52:09.512989 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a49d9e-cce2-494a-a4e9-6fbc9e59d35d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:52:09.524847 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a49d9e-cce2-494a-a4e9-6fbc9e59d35d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:52:09.527202 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a49d9e-cce2-494a-a4e9-6fbc9e59d35d service cinder] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:52:09.563778 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1370/2760] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:52:09 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:52:09.657303 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f030176-6141-4f06-988b-a184bcecf81d None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:52:09.658974 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-2f030176-6141-4f06-988b-a184bcecf81d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:52:09.680317 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f030176-6141-4f06-988b-a184bcecf81d None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:52:09.690405 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f127d11a-c4bc-4814-ac90-4a66e36a50fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24d5e2637b7043e08313909d5b06e329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:52:09.692980 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f127d11a-c4bc-4814-ac90-4a66e36a50fe service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:52:09.693468 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f127d11a-c4bc-4814-ac90-4a66e36a50fe service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:52:09.694042 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f127d11a-c4bc-4814-ac90-4a66e36a50fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:52:09.755938 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:52:09.827046 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b66bd5-8923-43ae-8cfb-16f24d268d7c None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:52:09.827474 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b66bd5-8923-43ae-8cfb-16f24d268d7c None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:52:09.827612 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b66bd5-8923-43ae-8cfb-16f24d268d7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:52:09.838482 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f127d11a-c4bc-4814-ac90-4a66e36a50fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:52:09.839435 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f127d11a-c4bc-4814-ac90-4a66e36a50fe service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:52:09.863790 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-31b66bd5-8923-43ae-8cfb-16f24d268d7c None None] MFA Rules not processed for user `a6da6754c62c4da6bb4a264456d2cf01`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:52:09.864561 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1391/2761] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:52:09 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:52:09.865176 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-31b66bd5-8923-43ae-8cfb-16f24d268d7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:52:09.992817 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:52:10.005043 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1371/2762] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:52:09 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:52:10.023492 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f127d11a-c4bc-4814-ac90-4a66e36a50fe service cinder] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:52:10.025170 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-f127d11a-c4bc-4814-ac90-4a66e36a50fe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:52:10.171985 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f127d11a-c4bc-4814-ac90-4a66e36a50fe service cinder] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:52:10.183549 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc809df2-a6a4-41ef-be58-88cc6c23b2d1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f86c16179a70474a817be9fda33ece9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:52:10.184591 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc809df2-a6a4-41ef-be58-88cc6c23b2d1 service nova] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:52:10.184933 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc809df2-a6a4-41ef-be58-88cc6c23b2d1 service nova] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:52:10.185276 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc809df2-a6a4-41ef-be58-88cc6c23b2d1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:52:10.189135 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-fc809df2-a6a4-41ef-be58-88cc6c23b2d1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:52:10.280737 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc809df2-a6a4-41ef-be58-88cc6c23b2d1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:52:10.282983 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc809df2-a6a4-41ef-be58-88cc6c23b2d1 service nova] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:52:10.298687 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1392/2763] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:52:10 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:52:14.272860 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b66bd5-8923-43ae-8cfb-16f24d268d7c None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:52:14.394523 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b66bd5-8923-43ae-8cfb-16f24d268d7c None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:52:14.401408 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52266f0-7845-4911-8c44-9c8fc0182ce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:52:14.402931 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52266f0-7845-4911-8c44-9c8fc0182ce5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:52:14.403506 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52266f0-7845-4911-8c44-9c8fc0182ce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:52:14.404002 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52266f0-7845-4911-8c44-9c8fc0182ce5 admin admin] PATH_INFO: `/v3/users/d47e4a43c01d447f952544072c560e73` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:52:14.405704 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52266f0-7845-4911-8c44-9c8fc0182ce5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d47e4a43c01d447f952544072c560e73)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:52:14.413849 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52266f0-7845-4911-8c44-9c8fc0182ce5 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:52:14.491354 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-a52266f0-7845-4911-8c44-9c8fc0182ce5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd47e4a43c01d447f952544072c560e73'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:52:14.492452 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-a52266f0-7845-4911-8c44-9c8fc0182ce5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd47e4a43c01d447f952544072c560e73'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:52:14.517182 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-a52266f0-7845-4911-8c44-9c8fc0182ce5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd47e4a43c01d447f952544072c560e73'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:52:14.518104 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-a52266f0-7845-4911-8c44-9c8fc0182ce5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd47e4a43c01d447f952544072c560e73'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:52:14.542863 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-a52266f0-7845-4911-8c44-9c8fc0182ce5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd47e4a43c01d447f952544072c560e73'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:52:14.569450 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1372/2764] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jun 16 17:52:14 2022] DELETE /identity/v3/users/d47e4a43c01d447f952544072c560e73 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:52:14.586045 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ca569a-eeaa-43d6-aab7-d781a1137ea1 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:52:14.586638 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ca569a-eeaa-43d6-aab7-d781a1137ea1 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:52:14.586853 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ca569a-eeaa-43d6-aab7-d781a1137ea1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:52:14.622895 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-04ca569a-eeaa-43d6-aab7-d781a1137ea1 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:52:14.624808 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-04ca569a-eeaa-43d6-aab7-d781a1137ea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:52:14.793766 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1393/2765] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:52:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:52:14.804660 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52266f0-7845-4911-8c44-9c8fc0182ce5 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:52:14.806155 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-a52266f0-7845-4911-8c44-9c8fc0182ce5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:52:14.897590 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52266f0-7845-4911-8c44-9c8fc0182ce5 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:52:14.914210 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982881b7-1e8f-43f5-9db9-7bcaabff72b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:52:14.915496 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982881b7-1e8f-43f5-9db9-7bcaabff72b3 service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:52:14.915953 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982881b7-1e8f-43f5-9db9-7bcaabff72b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:52:14.917065 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982881b7-1e8f-43f5-9db9-7bcaabff72b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:52:14.930953 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982881b7-1e8f-43f5-9db9-7bcaabff72b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:52:14.932542 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982881b7-1e8f-43f5-9db9-7bcaabff72b3 service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:52:14.951195 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1373/2766] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:52:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:52:15.321571 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ca569a-eeaa-43d6-aab7-d781a1137ea1 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:52:15.323290 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-04ca569a-eeaa-43d6-aab7-d781a1137ea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:52:15.332008 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:52:15.338953 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ca569a-eeaa-43d6-aab7-d781a1137ea1 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:52:15.357132 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bf2f02-dc2d-4fe9-bab3-983a1e252330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:52:15.359607 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bf2f02-dc2d-4fe9-bab3-983a1e252330 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:52:15.360053 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bf2f02-dc2d-4fe9-bab3-983a1e252330 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:52:15.360774 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bf2f02-dc2d-4fe9-bab3-983a1e252330 admin admin] PATH_INFO: `/v3/projects/873d76fb7094436ea3c5123aa959a0da` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:52:15.362256 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bf2f02-dc2d-4fe9-bab3-983a1e252330 admin admin] RBAC: Authorizing `identity:delete_project(project_id=873d76fb7094436ea3c5123aa959a0da)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:52:15.365168 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bf2f02-dc2d-4fe9-bab3-983a1e252330 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:52:15.451388 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-e1bf2f02-dc2d-4fe9-bab3-983a1e252330 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '873d76fb7094436ea3c5123aa959a0da', 'request_id': 'req-e1bf2f02-dc2d-4fe9-bab3-983a1e252330'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:52:15.461827 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-e1bf2f02-dc2d-4fe9-bab3-983a1e252330 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '873d76fb7094436ea3c5123aa959a0da', 'request_id': 'req-e1bf2f02-dc2d-4fe9-bab3-983a1e252330'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:52:15.473565 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-e1bf2f02-dc2d-4fe9-bab3-983a1e252330 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '873d76fb7094436ea3c5123aa959a0da', 'request_id': 'req-e1bf2f02-dc2d-4fe9-bab3-983a1e252330'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:52:15.490310 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-e1bf2f02-dc2d-4fe9-bab3-983a1e252330 admin admin] The token cache is being invalidate because project 873d76fb7094436ea3c5123aa959a0da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:52:15.490913 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-e1bf2f02-dc2d-4fe9-bab3-983a1e252330 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:52:15.491807 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-e1bf2f02-dc2d-4fe9-bab3-983a1e252330 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:52:15.493858 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1394/2767] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jun 16 17:52:15 2022] DELETE /identity/v3/projects/873d76fb7094436ea3c5123aa959a0da => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:52:15.497895 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982881b7-1e8f-43f5-9db9-7bcaabff72b3 service neutron] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:52:15.509150 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982881b7-1e8f-43f5-9db9-7bcaabff72b3 service neutron] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:52:15.548903 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:52:15.606100 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136bf61b-d6a3-4b57-8183-c5f504e1e22e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:52:15.607437 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136bf61b-d6a3-4b57-8183-c5f504e1e22e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:52:15.607796 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136bf61b-d6a3-4b57-8183-c5f504e1e22e admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:52:15.608355 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136bf61b-d6a3-4b57-8183-c5f504e1e22e admin admin] PATH_INFO: `/v3/users/a9149060f18246378e9fa928d055eb2a` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:52:15.610460 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136bf61b-d6a3-4b57-8183-c5f504e1e22e admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9149060f18246378e9fa928d055eb2a)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:52:15.614353 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136bf61b-d6a3-4b57-8183-c5f504e1e22e admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:52:15.672013 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-136bf61b-d6a3-4b57-8183-c5f504e1e22e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9149060f18246378e9fa928d055eb2a'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:52:15.672691 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-136bf61b-d6a3-4b57-8183-c5f504e1e22e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9149060f18246378e9fa928d055eb2a'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:52:15.681819 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-136bf61b-d6a3-4b57-8183-c5f504e1e22e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9149060f18246378e9fa928d055eb2a'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:52:15.682508 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-136bf61b-d6a3-4b57-8183-c5f504e1e22e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9149060f18246378e9fa928d055eb2a'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:52:15.696127 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-136bf61b-d6a3-4b57-8183-c5f504e1e22e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9149060f18246378e9fa928d055eb2a'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:52:15.712984 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1374/2768] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jun 16 17:52:15 2022] DELETE /identity/v3/users/a9149060f18246378e9fa928d055eb2a => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:52:16.064822 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bf2f02-dc2d-4fe9-bab3-983a1e252330 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:52:16.067726 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-e1bf2f02-dc2d-4fe9-bab3-983a1e252330 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:52:16.090983 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bf2f02-dc2d-4fe9-bab3-983a1e252330 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:52:16.166812 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f13636-ef3f-47b5-9b47-9f71283692be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:52:16.168123 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f13636-ef3f-47b5-9b47-9f71283692be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:52:16.168973 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f13636-ef3f-47b5-9b47-9f71283692be admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:52:16.169263 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f13636-ef3f-47b5-9b47-9f71283692be admin admin] PATH_INFO: `/v3/projects/c1ad546c304c494d8d7e455456817394` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:52:16.170875 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f13636-ef3f-47b5-9b47-9f71283692be admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1ad546c304c494d8d7e455456817394)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:52:16.173171 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f13636-ef3f-47b5-9b47-9f71283692be admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:52:16.229893 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-f6f13636-ef3f-47b5-9b47-9f71283692be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1ad546c304c494d8d7e455456817394', 'request_id': 'req-f6f13636-ef3f-47b5-9b47-9f71283692be'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:52:16.239598 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-f6f13636-ef3f-47b5-9b47-9f71283692be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1ad546c304c494d8d7e455456817394', 'request_id': 'req-f6f13636-ef3f-47b5-9b47-9f71283692be'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:52:16.248856 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-f6f13636-ef3f-47b5-9b47-9f71283692be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1ad546c304c494d8d7e455456817394', 'request_id': 'req-f6f13636-ef3f-47b5-9b47-9f71283692be'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:52:16.262034 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-f6f13636-ef3f-47b5-9b47-9f71283692be admin admin] The token cache is being invalidate because project c1ad546c304c494d8d7e455456817394 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:52:16.262855 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-f6f13636-ef3f-47b5-9b47-9f71283692be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:52:16.263637 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-f6f13636-ef3f-47b5-9b47-9f71283692be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:52:16.265854 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1395/2769] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jun 16 17:52:16 2022] DELETE /identity/v3/projects/c1ad546c304c494d8d7e455456817394 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:52:16.273865 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f85a0b2-1809-43ce-89ed-0a37a8f770d0 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:52:16.274322 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f85a0b2-1809-43ce-89ed-0a37a8f770d0 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:52:16.274799 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f85a0b2-1809-43ce-89ed-0a37a8f770d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:52:16.297370 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-7f85a0b2-1809-43ce-89ed-0a37a8f770d0 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:52:16.298605 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-7f85a0b2-1809-43ce-89ed-0a37a8f770d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:52:16.422986 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1375/2770] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:52:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:52:16.424660 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8472addf-a8ba-4f3c-b5d5-e3ce2c86e7de None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:52:16.425043 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8472addf-a8ba-4f3c-b5d5-e3ce2c86e7de None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:52:16.425361 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8472addf-a8ba-4f3c-b5d5-e3ce2c86e7de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:52:16.447244 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-8472addf-a8ba-4f3c-b5d5-e3ce2c86e7de None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:52:16.449162 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-8472addf-a8ba-4f3c-b5d5-e3ce2c86e7de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:52:16.459132 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1396/2771] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:52:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:52:16.464016 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f85a0b2-1809-43ce-89ed-0a37a8f770d0 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:52:16.476305 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f85a0b2-1809-43ce-89ed-0a37a8f770d0 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:52:16.481353 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268ab8c4-3946-4ad5-9f45-99ea51a37c22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:52:16.482042 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268ab8c4-3946-4ad5-9f45-99ea51a37c22 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:52:16.482330 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268ab8c4-3946-4ad5-9f45-99ea51a37c22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:52:16.482623 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268ab8c4-3946-4ad5-9f45-99ea51a37c22 admin admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:52:16.483379 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268ab8c4-3946-4ad5-9f45-99ea51a37c22 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:52:16.483696 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268ab8c4-3946-4ad5-9f45-99ea51a37c22 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:52:16.485509 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268ab8c4-3946-4ad5-9f45-99ea51a37c22 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:52:16.508407 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1376/2772] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jun 16 17:52:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:52:16.511617 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8472addf-a8ba-4f3c-b5d5-e3ce2c86e7de None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:52:16.526486 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8472addf-a8ba-4f3c-b5d5-e3ce2c86e7de None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:52:16.532593 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c440817-5ccc-43e2-bff2-1dc336108663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:52:16.533440 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c440817-5ccc-43e2-bff2-1dc336108663 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:52:16.536621 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c440817-5ccc-43e2-bff2-1dc336108663 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:52:16.537040 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c440817-5ccc-43e2-bff2-1dc336108663 admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:52:16.537915 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c440817-5ccc-43e2-bff2-1dc336108663 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:52:16.539970 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c440817-5ccc-43e2-bff2-1dc336108663 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:52:16.545017 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:52:16.577873 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1397/2773] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jun 16 17:52:16 2022] POST /identity/v3/projects => generated 409 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:52:16.582363 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268ab8c4-3946-4ad5-9f45-99ea51a37c22 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:52:16.593278 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268ab8c4-3946-4ad5-9f45-99ea51a37c22 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:52:16.628832 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:52:16.667090 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1161c1b3-6a30-40ff-9b3c-322661991df7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:52:16.668188 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1161c1b3-6a30-40ff-9b3c-322661991df7 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:52:16.668634 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1161c1b3-6a30-40ff-9b3c-322661991df7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:52:16.668971 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1161c1b3-6a30-40ff-9b3c-322661991df7 admin admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:52:16.669864 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1161c1b3-6a30-40ff-9b3c-322661991df7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:52:16.672978 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1161c1b3-6a30-40ff-9b3c-322661991df7 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:52:16.701661 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1377/2774] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jun 16 17:52:16 2022] POST /identity/v3/users => generated 305 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:52:16.706600 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c440817-5ccc-43e2-bff2-1dc336108663 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:52:16.721562 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c440817-5ccc-43e2-bff2-1dc336108663 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:52:16.726523 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61476c8-850c-4cd6-ba10-8bf4b7db66db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:52:16.727196 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61476c8-850c-4cd6-ba10-8bf4b7db66db admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:52:16.727482 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61476c8-850c-4cd6-ba10-8bf4b7db66db admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:52:16.727764 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61476c8-850c-4cd6-ba10-8bf4b7db66db admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:52:16.728541 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61476c8-850c-4cd6-ba10-8bf4b7db66db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:52:16.730166 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61476c8-850c-4cd6-ba10-8bf4b7db66db admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:52:16.742003 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1398/2775] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:52:16 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 17:52:16.746425 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1161c1b3-6a30-40ff-9b3c-322661991df7 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:52:16.758076 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1161c1b3-6a30-40ff-9b3c-322661991df7 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:52:16.763308 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b807a2-0241-4c4c-9a5f-206169ec5570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:52:16.763981 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b807a2-0241-4c4c-9a5f-206169ec5570 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:52:16.764297 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b807a2-0241-4c4c-9a5f-206169ec5570 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:52:16.764614 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b807a2-0241-4c4c-9a5f-206169ec5570 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:52:16.765762 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b807a2-0241-4c4c-9a5f-206169ec5570 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:52:16.767401 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b807a2-0241-4c4c-9a5f-206169ec5570 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:52:16.779902 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1378/2776] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:52:16 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:52:16.783990 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61476c8-850c-4cd6-ba10-8bf4b7db66db admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:52:16.796217 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61476c8-850c-4cd6-ba10-8bf4b7db66db admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:52:16.807126 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373a3f93-2165-4a96-b024-4423798cfd07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:52:16.808086 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373a3f93-2165-4a96-b024-4423798cfd07 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:52:16.808388 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373a3f93-2165-4a96-b024-4423798cfd07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:52:16.808692 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373a3f93-2165-4a96-b024-4423798cfd07 admin admin] PATH_INFO: `/v3/projects/983e83f363f44dc985f61c55452a58f9/users/27161fcab5bd4e02932c39deb232a5ed/roles/0be2075946f54c8485f1c275f746478a` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:52:16.829645 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373a3f93-2165-4a96-b024-4423798cfd07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=983e83f363f44dc985f61c55452a58f9, user_id=27161fcab5bd4e02932c39deb232a5ed, role_id=0be2075946f54c8485f1c275f746478a)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:52:16.837611 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373a3f93-2165-4a96-b024-4423798cfd07 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:52:16.859534 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1399/2777] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:52:16 2022] PUT /identity/v3/projects/983e83f363f44dc985f61c55452a58f9/users/27161fcab5bd4e02932c39deb232a5ed/roles/0be2075946f54c8485f1c275f746478a => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:52:16.871352 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d695c424-8d66-4c9b-9e41-0dd38b3373ac None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:52:16.871697 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d695c424-8d66-4c9b-9e41-0dd38b3373ac None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:52:16.871928 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d695c424-8d66-4c9b-9e41-0dd38b3373ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:52:16.913452 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-d695c424-8d66-4c9b-9e41-0dd38b3373ac None None] MFA Rules not processed for user `27161fcab5bd4e02932c39deb232a5ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:52:16.914903 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-d695c424-8d66-4c9b-9e41-0dd38b3373ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:52:17.006066 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1379/2778] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:52:16 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:52:17.010736 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373a3f93-2165-4a96-b024-4423798cfd07 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:52:17.097833 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373a3f93-2165-4a96-b024-4423798cfd07 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:52:17.106423 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60003e10-2dfa-4740-a7fc-0dcc99ddaaef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:52:17.107176 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60003e10-2dfa-4740-a7fc-0dcc99ddaaef admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:52:17.107554 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60003e10-2dfa-4740-a7fc-0dcc99ddaaef admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:52:17.108179 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60003e10-2dfa-4740-a7fc-0dcc99ddaaef admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:52:17.109477 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60003e10-2dfa-4740-a7fc-0dcc99ddaaef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:52:17.111972 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60003e10-2dfa-4740-a7fc-0dcc99ddaaef admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:52:17.153335 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1400/2779] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jun 16 17:52:17 2022] POST /identity/v3/projects => generated 409 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:52:17.158085 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d695c424-8d66-4c9b-9e41-0dd38b3373ac None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:52:17.183082 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d695c424-8d66-4c9b-9e41-0dd38b3373ac None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:52:17.276942 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d146640e-8007-4cd0-b555-940f1571cfc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:52:17.277749 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d146640e-8007-4cd0-b555-940f1571cfc5 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:52:17.278043 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d146640e-8007-4cd0-b555-940f1571cfc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:52:17.278323 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d146640e-8007-4cd0-b555-940f1571cfc5 admin admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:52:17.279683 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d146640e-8007-4cd0-b555-940f1571cfc5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:52:17.287630 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d146640e-8007-4cd0-b555-940f1571cfc5 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:52:17.314090 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1380/2780] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jun 16 17:52:17 2022] POST /identity/v3/users => generated 311 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:52:17.319371 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60003e10-2dfa-4740-a7fc-0dcc99ddaaef admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:52:17.332425 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60003e10-2dfa-4740-a7fc-0dcc99ddaaef admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:52:17.337532 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14e4e2b-1e67-47d4-9fb7-858b69ff42b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:52:17.338433 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14e4e2b-1e67-47d4-9fb7-858b69ff42b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:52:17.338881 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14e4e2b-1e67-47d4-9fb7-858b69ff42b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:52:17.339268 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14e4e2b-1e67-47d4-9fb7-858b69ff42b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:52:17.340089 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14e4e2b-1e67-47d4-9fb7-858b69ff42b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:52:17.341867 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14e4e2b-1e67-47d4-9fb7-858b69ff42b2 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:52:17.356512 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1401/2781] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:52:17 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:52:17.357975 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d146640e-8007-4cd0-b555-940f1571cfc5 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:52:17.368608 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d146640e-8007-4cd0-b555-940f1571cfc5 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:52:17.373328 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7440ac-3f53-4344-b046-492f1f3232ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:52:17.374327 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7440ac-3f53-4344-b046-492f1f3232ed admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:52:17.374706 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7440ac-3f53-4344-b046-492f1f3232ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:52:17.375194 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7440ac-3f53-4344-b046-492f1f3232ed admin admin] PATH_INFO: `/v3/projects/3b68ea58a38c49088124f0f069d2afda/users/a5fef287aadc46f8891736568c6da72a/roles/3ca2dcbbd29d4a31a3ab78f2299bc239` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:52:17.393030 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7440ac-3f53-4344-b046-492f1f3232ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b68ea58a38c49088124f0f069d2afda, user_id=a5fef287aadc46f8891736568c6da72a, role_id=3ca2dcbbd29d4a31a3ab78f2299bc239)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:52:17.395927 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7440ac-3f53-4344-b046-492f1f3232ed admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:52:17.410468 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1381/2782] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:52:17 2022] PUT /identity/v3/projects/3b68ea58a38c49088124f0f069d2afda/users/a5fef287aadc46f8891736568c6da72a/roles/3ca2dcbbd29d4a31a3ab78f2299bc239 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:52:17.414762 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14e4e2b-1e67-47d4-9fb7-858b69ff42b2 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:52:17.426561 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14e4e2b-1e67-47d4-9fb7-858b69ff42b2 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:52:17.495453 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84616df6-a0c2-4c12-a09a-88048a5d1c1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:52:17.496161 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84616df6-a0c2-4c12-a09a-88048a5d1c1d admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:52:17.496480 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84616df6-a0c2-4c12-a09a-88048a5d1c1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:52:17.496766 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84616df6-a0c2-4c12-a09a-88048a5d1c1d admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:52:17.497491 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84616df6-a0c2-4c12-a09a-88048a5d1c1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:52:17.499096 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84616df6-a0c2-4c12-a09a-88048a5d1c1d admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:52:17.511014 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1402/2783] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:52:17 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:52:17.515702 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7440ac-3f53-4344-b046-492f1f3232ed admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:52:17.521478 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:52:17.529107 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7440ac-3f53-4344-b046-492f1f3232ed admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:52:17.534393 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8363a0f-d7e6-4a82-ac0b-39ca623de187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:52:17.535311 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8363a0f-d7e6-4a82-ac0b-39ca623de187 admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:52:17.535608 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8363a0f-d7e6-4a82-ac0b-39ca623de187 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:52:17.535890 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8363a0f-d7e6-4a82-ac0b-39ca623de187 admin admin] PATH_INFO: `/v3/projects/3b68ea58a38c49088124f0f069d2afda/users/a5fef287aadc46f8891736568c6da72a/roles/3ca2dcbbd29d4a31a3ab78f2299bc239` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:52:17.538399 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8363a0f-d7e6-4a82-ac0b-39ca623de187 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b68ea58a38c49088124f0f069d2afda, user_id=a5fef287aadc46f8891736568c6da72a, role_id=3ca2dcbbd29d4a31a3ab78f2299bc239)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:52:17.541679 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8363a0f-d7e6-4a82-ac0b-39ca623de187 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:52:17.555148 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1382/2784] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:52:17 2022] PUT /identity/v3/projects/3b68ea58a38c49088124f0f069d2afda/users/a5fef287aadc46f8891736568c6da72a/roles/3ca2dcbbd29d4a31a3ab78f2299bc239 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:52:17.566487 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f0d29d-be9e-4287-9f14-8573ccf2357e None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:52:17.566796 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f0d29d-be9e-4287-9f14-8573ccf2357e None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:52:17.567086 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f0d29d-be9e-4287-9f14-8573ccf2357e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:52:17.595623 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-29f0d29d-be9e-4287-9f14-8573ccf2357e None None] MFA Rules not processed for user `a5fef287aadc46f8891736568c6da72a`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:52:17.597295 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-29f0d29d-be9e-4287-9f14-8573ccf2357e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:52:17.610119 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:52:17.689232 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1403/2785] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:52:17 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:52:17.701579 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8363a0f-d7e6-4a82-ac0b-39ca623de187 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:52:17.703518 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-e8363a0f-d7e6-4a82-ac0b-39ca623de187 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:52:17.781462 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8363a0f-d7e6-4a82-ac0b-39ca623de187 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:52:17.786100 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa675ad-b3c6-4ba3-b762-1efbf8bea684 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24d5e2637b7043e08313909d5b06e329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:52:17.786858 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa675ad-b3c6-4ba3-b762-1efbf8bea684 service cinder] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:52:17.787203 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa675ad-b3c6-4ba3-b762-1efbf8bea684 service cinder] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:52:17.787553 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa675ad-b3c6-4ba3-b762-1efbf8bea684 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:52:17.797342 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa675ad-b3c6-4ba3-b762-1efbf8bea684 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:52:17.798817 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa675ad-b3c6-4ba3-b762-1efbf8bea684 service cinder] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:52:17.809749 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1383/2786] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:52:17 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:52:18.013571 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f0d29d-be9e-4287-9f14-8573ccf2357e None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:52:18.015292 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-29f0d29d-be9e-4287-9f14-8573ccf2357e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:52:18.031277 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f0d29d-be9e-4287-9f14-8573ccf2357e None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:52:18.036185 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86427e28-90f9-4679-8b51-f4080a8ddad4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24d5e2637b7043e08313909d5b06e329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:52:18.036905 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86427e28-90f9-4679-8b51-f4080a8ddad4 service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:52:18.037198 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86427e28-90f9-4679-8b51-f4080a8ddad4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:52:18.037499 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86427e28-90f9-4679-8b51-f4080a8ddad4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:52:18.102230 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86427e28-90f9-4679-8b51-f4080a8ddad4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:52:18.103852 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86427e28-90f9-4679-8b51-f4080a8ddad4 service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:52:18.118825 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1404/2787] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:52:18 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 17:52:27.805191 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:52:28.111491 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=52 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:52:28.751598 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa675ad-b3c6-4ba3-b762-1efbf8bea684 service cinder] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:52:28.819298 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa675ad-b3c6-4ba3-b762-1efbf8bea684 service cinder] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:52:28.823672 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b03f3d-3cac-4096-86c3-3e723c103971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:52:28.824561 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b03f3d-3cac-4096-86c3-3e723c103971 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:52:28.824841 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b03f3d-3cac-4096-86c3-3e723c103971 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:52:28.825113 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b03f3d-3cac-4096-86c3-3e723c103971 admin admin] PATH_INFO: `/v3/users/27161fcab5bd4e02932c39deb232a5ed` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:52:28.826270 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b03f3d-3cac-4096-86c3-3e723c103971 admin admin] RBAC: Authorizing `identity:delete_user(user_id=27161fcab5bd4e02932c39deb232a5ed)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:52:28.828162 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b03f3d-3cac-4096-86c3-3e723c103971 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:52:28.879137 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-57b03f3d-3cac-4096-86c3-3e723c103971 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27161fcab5bd4e02932c39deb232a5ed'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:52:28.880303 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-57b03f3d-3cac-4096-86c3-3e723c103971 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27161fcab5bd4e02932c39deb232a5ed'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:52:28.889154 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-57b03f3d-3cac-4096-86c3-3e723c103971 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27161fcab5bd4e02932c39deb232a5ed'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:52:28.889774 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-57b03f3d-3cac-4096-86c3-3e723c103971 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27161fcab5bd4e02932c39deb232a5ed'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:52:28.902670 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-57b03f3d-3cac-4096-86c3-3e723c103971 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27161fcab5bd4e02932c39deb232a5ed'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:52:28.916777 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1384/2788] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jun 16 17:52:28 2022] DELETE /identity/v3/users/27161fcab5bd4e02932c39deb232a5ed => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:52:28.921464 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb30c249-692b-4bed-8ceb-ec99777ef4da None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:52:28.921769 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb30c249-692b-4bed-8ceb-ec99777ef4da None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:52:28.922056 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb30c249-692b-4bed-8ceb-ec99777ef4da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:52:28.941624 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-bb30c249-692b-4bed-8ceb-ec99777ef4da None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:52:28.946659 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-bb30c249-692b-4bed-8ceb-ec99777ef4da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:52:29.011047 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1405/2789] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:52:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:52:29.022208 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b03f3d-3cac-4096-86c3-3e723c103971 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:52:29.023624 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-57b03f3d-3cac-4096-86c3-3e723c103971 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:52:29.091008 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b03f3d-3cac-4096-86c3-3e723c103971 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:52:29.094813 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36910d6-6803-405a-8533-ca437c9c4e65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:52:29.095477 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36910d6-6803-405a-8533-ca437c9c4e65 service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:52:29.095760 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36910d6-6803-405a-8533-ca437c9c4e65 service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:52:29.096033 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36910d6-6803-405a-8533-ca437c9c4e65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:52:29.107042 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36910d6-6803-405a-8533-ca437c9c4e65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:52:29.108503 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36910d6-6803-405a-8533-ca437c9c4e65 service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:52:29.119217 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1385/2790] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:52:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:52:29.391626 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb30c249-692b-4bed-8ceb-ec99777ef4da None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:52:29.393333 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-bb30c249-692b-4bed-8ceb-ec99777ef4da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:52:29.406974 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb30c249-692b-4bed-8ceb-ec99777ef4da None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:52:29.411701 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f8b929-d42c-40d2-a5ff-cb940f7edba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:52:29.412584 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f8b929-d42c-40d2-a5ff-cb940f7edba5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:52:29.413394 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f8b929-d42c-40d2-a5ff-cb940f7edba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:52:29.413679 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f8b929-d42c-40d2-a5ff-cb940f7edba5 admin admin] PATH_INFO: `/v3/projects/983e83f363f44dc985f61c55452a58f9` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:52:29.415045 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f8b929-d42c-40d2-a5ff-cb940f7edba5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=983e83f363f44dc985f61c55452a58f9)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:52:29.417003 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f8b929-d42c-40d2-a5ff-cb940f7edba5 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:52:29.467111 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-83f8b929-d42c-40d2-a5ff-cb940f7edba5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '983e83f363f44dc985f61c55452a58f9', 'request_id': 'req-83f8b929-d42c-40d2-a5ff-cb940f7edba5'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:52:29.475436 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-83f8b929-d42c-40d2-a5ff-cb940f7edba5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '983e83f363f44dc985f61c55452a58f9', 'request_id': 'req-83f8b929-d42c-40d2-a5ff-cb940f7edba5'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:52:29.483177 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-83f8b929-d42c-40d2-a5ff-cb940f7edba5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '983e83f363f44dc985f61c55452a58f9', 'request_id': 'req-83f8b929-d42c-40d2-a5ff-cb940f7edba5'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:52:29.496155 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-83f8b929-d42c-40d2-a5ff-cb940f7edba5 admin admin] The token cache is being invalidate because project 983e83f363f44dc985f61c55452a58f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:52:29.496483 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-83f8b929-d42c-40d2-a5ff-cb940f7edba5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:52:29.497169 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-83f8b929-d42c-40d2-a5ff-cb940f7edba5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:52:29.498908 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1406/2791] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jun 16 17:52:29 2022] DELETE /identity/v3/projects/983e83f363f44dc985f61c55452a58f9 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:52:29.502712 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36910d6-6803-405a-8533-ca437c9c4e65 service neutron] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:52:29.514863 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36910d6-6803-405a-8533-ca437c9c4e65 service neutron] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:52:29.534232 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:52:29.578566 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed619b59-4701-44b4-8cb3-debdd574e7f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:52:29.579479 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed619b59-4701-44b4-8cb3-debdd574e7f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:52:29.579922 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed619b59-4701-44b4-8cb3-debdd574e7f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:52:29.580208 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed619b59-4701-44b4-8cb3-debdd574e7f8 admin admin] PATH_INFO: `/v3/users/a5fef287aadc46f8891736568c6da72a` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:52:29.581458 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed619b59-4701-44b4-8cb3-debdd574e7f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5fef287aadc46f8891736568c6da72a)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:52:29.584060 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed619b59-4701-44b4-8cb3-debdd574e7f8 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:52:29.633903 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-ed619b59-4701-44b4-8cb3-debdd574e7f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5fef287aadc46f8891736568c6da72a'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:52:29.634584 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-ed619b59-4701-44b4-8cb3-debdd574e7f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5fef287aadc46f8891736568c6da72a'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:52:29.643003 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-ed619b59-4701-44b4-8cb3-debdd574e7f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5fef287aadc46f8891736568c6da72a'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:52:29.643638 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-ed619b59-4701-44b4-8cb3-debdd574e7f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5fef287aadc46f8891736568c6da72a'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:52:29.657873 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-ed619b59-4701-44b4-8cb3-debdd574e7f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5fef287aadc46f8891736568c6da72a'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:52:29.675046 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1386/2792] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jun 16 17:52:29 2022] DELETE /identity/v3/users/a5fef287aadc46f8891736568c6da72a => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:52:29.797471 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a232c33-7d69-47f6-a8bc-59497aca8590 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:52:29.797744 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a232c33-7d69-47f6-a8bc-59497aca8590 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:52:29.798054 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a232c33-7d69-47f6-a8bc-59497aca8590 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:52:29.816441 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-7a232c33-7d69-47f6-a8bc-59497aca8590 None None] MFA Rules not processed for user `71443c30b4554719b5549493bf3cc00c`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:52:29.818386 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-7a232c33-7d69-47f6-a8bc-59497aca8590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:52:29.884245 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1407/2793] 10.222.0.42 () {58 vars in 967 bytes} [Thu Jun 16 17:52:29 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:52:29.897807 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed619b59-4701-44b4-8cb3-debdd574e7f8 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:52:29.899409 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-ed619b59-4701-44b4-8cb3-debdd574e7f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:52:29.974902 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a232c33-7d69-47f6-a8bc-59497aca8590 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:52:29.976729 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-7a232c33-7d69-47f6-a8bc-59497aca8590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:52:29.988155 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed619b59-4701-44b4-8cb3-debdd574e7f8 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:52:29.994126 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed86b437-8dd4-4041-b497-cc904e841f78 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47b7a9cd615c41d6983a0007b475038c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:52:29.994801 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed86b437-8dd4-4041-b497-cc904e841f78 service swift] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:52:29.995083 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed86b437-8dd4-4041-b497-cc904e841f78 service swift] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:52:29.995357 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed86b437-8dd4-4041-b497-cc904e841f78 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:52:30.006354 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed86b437-8dd4-4041-b497-cc904e841f78 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:52:30.008297 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed86b437-8dd4-4041-b497-cc904e841f78 service swift] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:52:30.013585 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:52:30.018020 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1387/2794] 10.222.0.42 () {60 vars in 1384 bytes} [Thu Jun 16 17:52:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 17:52:30.063673 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a232c33-7d69-47f6-a8bc-59497aca8590 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:52:30.068672 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e7555e-d459-4aa1-bd57-abbefd293d93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:52:30.069663 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e7555e-d459-4aa1-bd57-abbefd293d93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:52:30.069950 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e7555e-d459-4aa1-bd57-abbefd293d93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:52:30.070626 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e7555e-d459-4aa1-bd57-abbefd293d93 admin admin] PATH_INFO: `/v3/projects/3b68ea58a38c49088124f0f069d2afda` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:52:30.071451 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e7555e-d459-4aa1-bd57-abbefd293d93 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b68ea58a38c49088124f0f069d2afda)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:52:30.073527 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e7555e-d459-4aa1-bd57-abbefd293d93 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:52:30.129213 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-a4e7555e-d459-4aa1-bd57-abbefd293d93 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b68ea58a38c49088124f0f069d2afda', 'request_id': 'req-a4e7555e-d459-4aa1-bd57-abbefd293d93'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:52:30.138089 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-a4e7555e-d459-4aa1-bd57-abbefd293d93 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b68ea58a38c49088124f0f069d2afda', 'request_id': 'req-a4e7555e-d459-4aa1-bd57-abbefd293d93'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:52:30.148657 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-a4e7555e-d459-4aa1-bd57-abbefd293d93 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b68ea58a38c49088124f0f069d2afda', 'request_id': 'req-a4e7555e-d459-4aa1-bd57-abbefd293d93'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:52:30.167931 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-a4e7555e-d459-4aa1-bd57-abbefd293d93 admin admin] The token cache is being invalidate because project 3b68ea58a38c49088124f0f069d2afda was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:52:30.168472 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-a4e7555e-d459-4aa1-bd57-abbefd293d93 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:52:30.169427 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-a4e7555e-d459-4aa1-bd57-abbefd293d93 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:52:30.171793 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1408/2795] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jun 16 17:52:29 2022] DELETE /identity/v3/projects/3b68ea58a38c49088124f0f069d2afda => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:52:30.179576 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c93dbe-c10f-48dc-aca9-d1aa5d38e301 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:52:30.179870 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c93dbe-c10f-48dc-aca9-d1aa5d38e301 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:52:30.180202 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c93dbe-c10f-48dc-aca9-d1aa5d38e301 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:52:30.200969 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-25c93dbe-c10f-48dc-aca9-d1aa5d38e301 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:52:30.202194 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-25c93dbe-c10f-48dc-aca9-d1aa5d38e301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:52:30.241214 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:52:30.297560 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1388/2796] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:52:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:52:30.299742 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb77456-b880-46c5-94e9-0baa1bd5f820 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:52:30.300037 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb77456-b880-46c5-94e9-0baa1bd5f820 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:52:30.300325 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb77456-b880-46c5-94e9-0baa1bd5f820 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:52:30.322485 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-9eb77456-b880-46c5-94e9-0baa1bd5f820 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:52:30.324223 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-9eb77456-b880-46c5-94e9-0baa1bd5f820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:52:30.341127 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1409/2797] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:52:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:52:30.341862 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c93dbe-c10f-48dc-aca9-d1aa5d38e301 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:52:30.356621 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c93dbe-c10f-48dc-aca9-d1aa5d38e301 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:52:30.363923 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f515df20-2cf7-4b20-a710-c5edf99d3f48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:52:30.365693 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f515df20-2cf7-4b20-a710-c5edf99d3f48 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:52:30.366048 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f515df20-2cf7-4b20-a710-c5edf99d3f48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:52:30.366399 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f515df20-2cf7-4b20-a710-c5edf99d3f48 admin admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:52:30.367213 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f515df20-2cf7-4b20-a710-c5edf99d3f48 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:52:30.367776 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f515df20-2cf7-4b20-a710-c5edf99d3f48 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:52:30.370164 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f515df20-2cf7-4b20-a710-c5edf99d3f48 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:52:30.387677 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1389/2798] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jun 16 17:52:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:52:30.389400 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb77456-b880-46c5-94e9-0baa1bd5f820 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:52:30.402592 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb77456-b880-46c5-94e9-0baa1bd5f820 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:52:30.407702 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a18d6c8-2e69-4e82-a5fe-3ad307251d33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:52:30.408630 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a18d6c8-2e69-4e82-a5fe-3ad307251d33 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:52:30.408993 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a18d6c8-2e69-4e82-a5fe-3ad307251d33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:52:30.409390 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a18d6c8-2e69-4e82-a5fe-3ad307251d33 admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:52:30.410161 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a18d6c8-2e69-4e82-a5fe-3ad307251d33 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:52:30.413590 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a18d6c8-2e69-4e82-a5fe-3ad307251d33 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:52:30.452401 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1410/2799] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jun 16 17:52:30 2022] POST /identity/v3/projects => generated 365 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:52:30.465309 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f515df20-2cf7-4b20-a710-c5edf99d3f48 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:52:30.468518 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f515df20-2cf7-4b20-a710-c5edf99d3f48 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:52:30.526685 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce465d75-5f56-4e2f-b443-731192ff43d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:52:30.527364 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce465d75-5f56-4e2f-b443-731192ff43d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:52:30.527653 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce465d75-5f56-4e2f-b443-731192ff43d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:52:30.527932 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce465d75-5f56-4e2f-b443-731192ff43d8 admin admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:52:30.528783 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce465d75-5f56-4e2f-b443-731192ff43d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:52:30.530757 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce465d75-5f56-4e2f-b443-731192ff43d8 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:52:30.557129 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1390/2800] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jun 16 17:52:30 2022] POST /identity/v3/users => generated 283 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:52:30.561388 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a18d6c8-2e69-4e82-a5fe-3ad307251d33 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:52:30.574538 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a18d6c8-2e69-4e82-a5fe-3ad307251d33 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:52:30.582059 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c20f5f8-fbd1-4208-adce-4fcd5a1a5265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:52:30.583175 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c20f5f8-fbd1-4208-adce-4fcd5a1a5265 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:52:30.583741 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c20f5f8-fbd1-4208-adce-4fcd5a1a5265 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:52:30.584202 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c20f5f8-fbd1-4208-adce-4fcd5a1a5265 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:52:30.585298 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c20f5f8-fbd1-4208-adce-4fcd5a1a5265 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:52:30.587253 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c20f5f8-fbd1-4208-adce-4fcd5a1a5265 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:52:30.600415 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1411/2801] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:52:30 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:52:30.602853 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce465d75-5f56-4e2f-b443-731192ff43d8 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:52:30.613894 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce465d75-5f56-4e2f-b443-731192ff43d8 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:52:30.618745 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dce079-5ca8-40a7-bb59-7b28d8374ee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:52:30.619408 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dce079-5ca8-40a7-bb59-7b28d8374ee2 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:52:30.619692 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dce079-5ca8-40a7-bb59-7b28d8374ee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:52:30.619973 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dce079-5ca8-40a7-bb59-7b28d8374ee2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:52:30.620739 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dce079-5ca8-40a7-bb59-7b28d8374ee2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:52:30.622376 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dce079-5ca8-40a7-bb59-7b28d8374ee2 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:52:30.633816 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1391/2802] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:52:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:52:30.638741 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c20f5f8-fbd1-4208-adce-4fcd5a1a5265 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:52:30.654018 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c20f5f8-fbd1-4208-adce-4fcd5a1a5265 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:52:30.659380 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb81b3f-7051-46f6-a9fb-726211323980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:52:30.660289 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb81b3f-7051-46f6-a9fb-726211323980 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:52:30.660551 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb81b3f-7051-46f6-a9fb-726211323980 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:52:30.660774 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb81b3f-7051-46f6-a9fb-726211323980 admin admin] PATH_INFO: `/v3/projects/c6f92f56dd1344ca899912c704308955/users/50a264704a234809a7084f1117354cef/roles/0be2075946f54c8485f1c275f746478a` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:52:30.685603 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb81b3f-7051-46f6-a9fb-726211323980 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6f92f56dd1344ca899912c704308955, user_id=50a264704a234809a7084f1117354cef, role_id=0be2075946f54c8485f1c275f746478a)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:52:30.689183 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb81b3f-7051-46f6-a9fb-726211323980 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:52:30.706419 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1412/2803] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:52:30 2022] PUT /identity/v3/projects/c6f92f56dd1344ca899912c704308955/users/50a264704a234809a7084f1117354cef/roles/0be2075946f54c8485f1c275f746478a => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:52:30.718254 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa007ba-1708-4064-baa3-b799b8e0f1e8 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:52:30.718780 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa007ba-1708-4064-baa3-b799b8e0f1e8 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:52:30.719180 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa007ba-1708-4064-baa3-b799b8e0f1e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:52:30.778474 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-5fa007ba-1708-4064-baa3-b799b8e0f1e8 None None] MFA Rules not processed for user `50a264704a234809a7084f1117354cef`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:52:30.779881 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-5fa007ba-1708-4064-baa3-b799b8e0f1e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:52:30.865540 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1392/2804] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:52:30 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:52:30.878141 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb81b3f-7051-46f6-a9fb-726211323980 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:52:30.881502 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-2bb81b3f-7051-46f6-a9fb-726211323980 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:52:30.942186 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb81b3f-7051-46f6-a9fb-726211323980 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:52:30.947063 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2ae4b2-71f3-48d3-a6ca-def4fc86b8c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24d5e2637b7043e08313909d5b06e329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:52:30.947824 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2ae4b2-71f3-48d3-a6ca-def4fc86b8c3 service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:52:30.948126 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2ae4b2-71f3-48d3-a6ca-def4fc86b8c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:52:30.948460 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2ae4b2-71f3-48d3-a6ca-def4fc86b8c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:52:30.957401 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2ae4b2-71f3-48d3-a6ca-def4fc86b8c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:52:30.958898 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2ae4b2-71f3-48d3-a6ca-def4fc86b8c3 service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:52:30.962762 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:52:30.970059 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1413/2805] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:52:30 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:52:30.970210 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa007ba-1708-4064-baa3-b799b8e0f1e8 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:52:30.971586 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-5fa007ba-1708-4064-baa3-b799b8e0f1e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:52:31.007341 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:52:31.049088 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa007ba-1708-4064-baa3-b799b8e0f1e8 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:52:31.055424 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ef3b89-5582-442a-ac71-b451a918aceb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0570d8d9508d4546ae897276f7d181d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:52:31.056212 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ef3b89-5582-442a-ac71-b451a918aceb service placement] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:52:31.056585 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ef3b89-5582-442a-ac71-b451a918aceb service placement] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:52:31.056890 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ef3b89-5582-442a-ac71-b451a918aceb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:52:31.058817 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-81ef3b89-5582-442a-ac71-b451a918aceb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:52:31.071803 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ef3b89-5582-442a-ac71-b451a918aceb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:52:31.073485 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ef3b89-5582-442a-ac71-b451a918aceb service placement] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:52:31.083592 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1393/2806] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:52:30 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 17:52:31.955450 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2ae4b2-71f3-48d3-a6ca-def4fc86b8c3 service cinder] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:52:31.957207 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-2a2ae4b2-71f3-48d3-a6ca-def4fc86b8c3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:52:31.980540 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2ae4b2-71f3-48d3-a6ca-def4fc86b8c3 service cinder] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:52:31.986707 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db8d8e8-0d5e-4a2e-97c4-8941d7971861 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0570d8d9508d4546ae897276f7d181d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:52:31.987955 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db8d8e8-0d5e-4a2e-97c4-8941d7971861 service placement] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:52:31.988308 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db8d8e8-0d5e-4a2e-97c4-8941d7971861 service placement] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:52:31.988702 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db8d8e8-0d5e-4a2e-97c4-8941d7971861 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:52:31.991102 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-3db8d8e8-0d5e-4a2e-97c4-8941d7971861 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:52:32.009273 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db8d8e8-0d5e-4a2e-97c4-8941d7971861 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:52:32.011316 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db8d8e8-0d5e-4a2e-97c4-8941d7971861 service placement] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:52:32.025693 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1414/2807] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:52:31 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:52:32.026606 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ef3b89-5582-442a-ac71-b451a918aceb service placement] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:52:32.028137 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-81ef3b89-5582-442a-ac71-b451a918aceb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:52:32.110396 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ef3b89-5582-442a-ac71-b451a918aceb service placement] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:52:32.115995 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629d74c4-086b-448b-ab86-162bda560e56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:52:32.116892 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629d74c4-086b-448b-ab86-162bda560e56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:52:32.117173 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629d74c4-086b-448b-ab86-162bda560e56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:52:32.117443 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629d74c4-086b-448b-ab86-162bda560e56 admin admin] PATH_INFO: `/v3/users/c4bffefe8fb2475a96ea033d07c790f2` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:52:32.118623 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629d74c4-086b-448b-ab86-162bda560e56 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4bffefe8fb2475a96ea033d07c790f2)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:52:32.120555 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629d74c4-086b-448b-ab86-162bda560e56 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:52:32.167570 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-629d74c4-086b-448b-ab86-162bda560e56 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4bffefe8fb2475a96ea033d07c790f2'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:52:32.168213 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-629d74c4-086b-448b-ab86-162bda560e56 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4bffefe8fb2475a96ea033d07c790f2'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:52:32.175565 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-629d74c4-086b-448b-ab86-162bda560e56 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4bffefe8fb2475a96ea033d07c790f2'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:52:32.176473 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-629d74c4-086b-448b-ab86-162bda560e56 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4bffefe8fb2475a96ea033d07c790f2'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:52:32.189053 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-629d74c4-086b-448b-ab86-162bda560e56 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4bffefe8fb2475a96ea033d07c790f2'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:52:32.201595 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1394/2808] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jun 16 17:52:32 2022] DELETE /identity/v3/users/c4bffefe8fb2475a96ea033d07c790f2 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:52:32.381582 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db8d8e8-0d5e-4a2e-97c4-8941d7971861 service placement] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:52:32.401661 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db8d8e8-0d5e-4a2e-97c4-8941d7971861 service placement] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:52:32.467536 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24a3780-2d3a-499e-ab6d-293b938a3c0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:52:32.468584 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24a3780-2d3a-499e-ab6d-293b938a3c0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:52:32.468876 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24a3780-2d3a-499e-ab6d-293b938a3c0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:52:32.469200 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24a3780-2d3a-499e-ab6d-293b938a3c0f admin admin] PATH_INFO: `/v3/projects/c6a903cccbd4459288d72e52410ffca1` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:52:32.470759 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24a3780-2d3a-499e-ab6d-293b938a3c0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6a903cccbd4459288d72e52410ffca1)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:52:32.472999 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24a3780-2d3a-499e-ab6d-293b938a3c0f admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:52:32.534340 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-b24a3780-2d3a-499e-ab6d-293b938a3c0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6a903cccbd4459288d72e52410ffca1', 'request_id': 'req-b24a3780-2d3a-499e-ab6d-293b938a3c0f'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:52:32.543409 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-b24a3780-2d3a-499e-ab6d-293b938a3c0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6a903cccbd4459288d72e52410ffca1', 'request_id': 'req-b24a3780-2d3a-499e-ab6d-293b938a3c0f'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:52:32.549968 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-b24a3780-2d3a-499e-ab6d-293b938a3c0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6a903cccbd4459288d72e52410ffca1', 'request_id': 'req-b24a3780-2d3a-499e-ab6d-293b938a3c0f'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:52:32.560684 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-b24a3780-2d3a-499e-ab6d-293b938a3c0f admin admin] The token cache is being invalidate because project c6a903cccbd4459288d72e52410ffca1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:52:32.560966 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-b24a3780-2d3a-499e-ab6d-293b938a3c0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:52:32.561618 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-b24a3780-2d3a-499e-ab6d-293b938a3c0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:52:32.563315 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1415/2809] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jun 16 17:52:32 2022] DELETE /identity/v3/projects/c6a903cccbd4459288d72e52410ffca1 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:52:32.568025 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bae70c-b910-4859-af84-95e1401d37c7 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:52:32.568402 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bae70c-b910-4859-af84-95e1401d37c7 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:52:32.569013 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bae70c-b910-4859-af84-95e1401d37c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:52:32.586756 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-19bae70c-b910-4859-af84-95e1401d37c7 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:52:32.587962 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-19bae70c-b910-4859-af84-95e1401d37c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:52:32.619591 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:52:32.653423 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1395/2810] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:52:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:52:32.661740 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417851a7-0387-464e-a09b-672f6b414dd6 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:52:32.662098 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417851a7-0387-464e-a09b-672f6b414dd6 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:52:32.662298 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417851a7-0387-464e-a09b-672f6b414dd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:52:32.684214 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-417851a7-0387-464e-a09b-672f6b414dd6 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:52:32.685576 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-417851a7-0387-464e-a09b-672f6b414dd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:52:32.696799 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1416/2811] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:52:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:52:32.704638 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bae70c-b910-4859-af84-95e1401d37c7 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:52:32.716522 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bae70c-b910-4859-af84-95e1401d37c7 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:52:32.721385 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4483089-79c8-43a7-bfa7-34a9e0ae613a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:52:32.722198 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4483089-79c8-43a7-bfa7-34a9e0ae613a admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:52:32.722482 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4483089-79c8-43a7-bfa7-34a9e0ae613a admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:52:32.722758 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4483089-79c8-43a7-bfa7-34a9e0ae613a admin admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:52:32.724298 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4483089-79c8-43a7-bfa7-34a9e0ae613a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:52:32.724775 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4483089-79c8-43a7-bfa7-34a9e0ae613a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:52:32.726631 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4483089-79c8-43a7-bfa7-34a9e0ae613a admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:52:32.738771 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1396/2812] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jun 16 17:52:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:52:32.744073 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417851a7-0387-464e-a09b-672f6b414dd6 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:52:32.755202 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417851a7-0387-464e-a09b-672f6b414dd6 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:52:32.760649 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4b3d7b-9142-44b6-8fdc-ff20f324683a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:52:32.761453 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4b3d7b-9142-44b6-8fdc-ff20f324683a admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:52:32.761988 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4b3d7b-9142-44b6-8fdc-ff20f324683a admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:52:32.762153 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4b3d7b-9142-44b6-8fdc-ff20f324683a admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:52:32.763142 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4b3d7b-9142-44b6-8fdc-ff20f324683a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:52:32.765115 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4b3d7b-9142-44b6-8fdc-ff20f324683a admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:52:32.799513 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1417/2813] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jun 16 17:52:32 2022] POST /identity/v3/projects => generated 371 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:52:32.804275 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4483089-79c8-43a7-bfa7-34a9e0ae613a admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:52:32.818060 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4483089-79c8-43a7-bfa7-34a9e0ae613a admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:52:32.891666 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0bfc4f-d894-45c8-8524-9267027e6bcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:52:32.892394 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0bfc4f-d894-45c8-8524-9267027e6bcb admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:52:32.892703 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0bfc4f-d894-45c8-8524-9267027e6bcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:52:32.892963 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0bfc4f-d894-45c8-8524-9267027e6bcb admin admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:52:32.893693 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0bfc4f-d894-45c8-8524-9267027e6bcb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:52:32.896821 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0bfc4f-d894-45c8-8524-9267027e6bcb admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:52:32.920984 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1397/2814] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jun 16 17:52:32 2022] POST /identity/v3/users => generated 286 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:52:32.922200 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4b3d7b-9142-44b6-8fdc-ff20f324683a admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:52:32.932590 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4b3d7b-9142-44b6-8fdc-ff20f324683a admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:52:32.937183 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5e6bad-0d06-40c1-af72-b8cd6147bd60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:52:32.937835 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5e6bad-0d06-40c1-af72-b8cd6147bd60 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:52:32.938117 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5e6bad-0d06-40c1-af72-b8cd6147bd60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:52:32.938392 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5e6bad-0d06-40c1-af72-b8cd6147bd60 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:52:32.939109 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5e6bad-0d06-40c1-af72-b8cd6147bd60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:52:32.940812 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5e6bad-0d06-40c1-af72-b8cd6147bd60 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:52:32.951715 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1418/2815] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:52:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 17:52:32.955804 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0bfc4f-d894-45c8-8524-9267027e6bcb admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:52:32.967015 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0bfc4f-d894-45c8-8524-9267027e6bcb admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:52:32.971655 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8483aab-9dbd-4473-98a6-66178f3927d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:52:32.972313 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8483aab-9dbd-4473-98a6-66178f3927d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:52:32.972633 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8483aab-9dbd-4473-98a6-66178f3927d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:52:32.972915 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8483aab-9dbd-4473-98a6-66178f3927d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:52:32.973808 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8483aab-9dbd-4473-98a6-66178f3927d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:52:32.975267 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8483aab-9dbd-4473-98a6-66178f3927d7 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:52:32.985313 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1398/2816] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:52:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:52:32.990019 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5e6bad-0d06-40c1-af72-b8cd6147bd60 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:52:32.999983 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5e6bad-0d06-40c1-af72-b8cd6147bd60 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:52:33.005377 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5526b777-a0df-43a8-884c-97493c9719d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:52:33.006504 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5526b777-a0df-43a8-884c-97493c9719d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:52:33.006916 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5526b777-a0df-43a8-884c-97493c9719d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:52:33.007202 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5526b777-a0df-43a8-884c-97493c9719d8 admin admin] PATH_INFO: `/v3/projects/2e60fcf741034efca613038cd64a9f2a/users/9284b0bf5ed84e98b503c64c6e7e6eac/roles/0be2075946f54c8485f1c275f746478a` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:52:33.022016 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5526b777-a0df-43a8-884c-97493c9719d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e60fcf741034efca613038cd64a9f2a, user_id=9284b0bf5ed84e98b503c64c6e7e6eac, role_id=0be2075946f54c8485f1c275f746478a)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:52:33.024694 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5526b777-a0df-43a8-884c-97493c9719d8 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:52:33.037174 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1419/2817] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:52:32 2022] PUT /identity/v3/projects/2e60fcf741034efca613038cd64a9f2a/users/9284b0bf5ed84e98b503c64c6e7e6eac/roles/0be2075946f54c8485f1c275f746478a => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:52:33.046968 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79ef4c0-c8cc-4d4e-8a8f-f1f96776c677 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:52:33.047480 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79ef4c0-c8cc-4d4e-8a8f-f1f96776c677 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:52:33.047579 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79ef4c0-c8cc-4d4e-8a8f-f1f96776c677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:52:33.080129 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-d79ef4c0-c8cc-4d4e-8a8f-f1f96776c677 None None] MFA Rules not processed for user `9284b0bf5ed84e98b503c64c6e7e6eac`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:52:33.081405 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-d79ef4c0-c8cc-4d4e-8a8f-f1f96776c677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:52:33.167354 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1399/2818] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:52:33 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:52:33.179436 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5526b777-a0df-43a8-884c-97493c9719d8 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:52:33.181133 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-5526b777-a0df-43a8-884c-97493c9719d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:52:33.195368 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:52:33.239905 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5526b777-a0df-43a8-884c-97493c9719d8 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:52:33.243759 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad94674-fe9f-4042-83b6-c59c4b0b3af5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd89029c1b3134df1962c041988bead74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:52:33.244429 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad94674-fe9f-4042-83b6-c59c4b0b3af5 service glance] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:52:33.244720 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad94674-fe9f-4042-83b6-c59c4b0b3af5 service glance] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:52:33.245000 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad94674-fe9f-4042-83b6-c59c4b0b3af5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:52:33.253306 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad94674-fe9f-4042-83b6-c59c4b0b3af5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:52:33.255492 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad94674-fe9f-4042-83b6-c59c4b0b3af5 service glance] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:52:33.264553 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1420/2819] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:52:33 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:52:41.720930 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39ae5d6-cf59-420c-82c0-aa9c402f3e81 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:52:41.721418 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39ae5d6-cf59-420c-82c0-aa9c402f3e81 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:52:41.721647 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39ae5d6-cf59-420c-82c0-aa9c402f3e81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:52:41.738760 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-c39ae5d6-cf59-420c-82c0-aa9c402f3e81 None None] MFA Rules not processed for user `0824d8df380e477b85690814b0a2a9e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:52:41.739960 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-c39ae5d6-cf59-420c-82c0-aa9c402f3e81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:52:41.768827 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:52:41.801458 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1400/2820] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:52:41 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:52:43.260287 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=21 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:52:47.594752 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ff1522-cde0-4ed7-a0d1-f74f8b164e02 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:52:47.595456 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ff1522-cde0-4ed7-a0d1-f74f8b164e02 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:52:47.595456 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ff1522-cde0-4ed7-a0d1-f74f8b164e02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:52:47.616872 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-b1ff1522-cde0-4ed7-a0d1-f74f8b164e02 None None] MFA Rules not processed for user `31e04db1b0f6444e9a884a0e58ba38da`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:52:47.618252 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-b1ff1522-cde0-4ed7-a0d1-f74f8b164e02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:52:47.696892 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1421/2821] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:52:47 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:52:51.793715 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=11 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:52:57.687549 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=25 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:53:16.581309 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39ae5d6-cf59-420c-82c0-aa9c402f3e81 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:53:16.649290 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39ae5d6-cf59-420c-82c0-aa9c402f3e81 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:53:16.653870 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9927073-c007-45df-9e1e-1ab68d76ca15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:53:16.654742 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9927073-c007-45df-9e1e-1ab68d76ca15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:53:16.655030 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9927073-c007-45df-9e1e-1ab68d76ca15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:53:16.655311 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9927073-c007-45df-9e1e-1ab68d76ca15 admin admin] PATH_INFO: `/v3/users/9284b0bf5ed84e98b503c64c6e7e6eac` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:53:16.656543 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9927073-c007-45df-9e1e-1ab68d76ca15 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9284b0bf5ed84e98b503c64c6e7e6eac)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:53:16.658474 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9927073-c007-45df-9e1e-1ab68d76ca15 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:53:16.703703 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-f9927073-c007-45df-9e1e-1ab68d76ca15 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9284b0bf5ed84e98b503c64c6e7e6eac'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:53:16.704393 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-f9927073-c007-45df-9e1e-1ab68d76ca15 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9284b0bf5ed84e98b503c64c6e7e6eac'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:53:16.713057 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-f9927073-c007-45df-9e1e-1ab68d76ca15 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9284b0bf5ed84e98b503c64c6e7e6eac'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:53:16.713720 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-f9927073-c007-45df-9e1e-1ab68d76ca15 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9284b0bf5ed84e98b503c64c6e7e6eac'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:53:16.726312 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-f9927073-c007-45df-9e1e-1ab68d76ca15 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9284b0bf5ed84e98b503c64c6e7e6eac'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:53:16.738450 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1401/2822] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jun 16 17:53:16 2022] DELETE /identity/v3/users/9284b0bf5ed84e98b503c64c6e7e6eac => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:53:16.743578 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ddaca5-0ee0-4f7b-8956-fcfba1f2f4d3 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:53:16.744109 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ddaca5-0ee0-4f7b-8956-fcfba1f2f4d3 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:53:16.744226 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ddaca5-0ee0-4f7b-8956-fcfba1f2f4d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:53:16.761450 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-f8ddaca5-0ee0-4f7b-8956-fcfba1f2f4d3 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:53:16.762912 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-f8ddaca5-0ee0-4f7b-8956-fcfba1f2f4d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:53:16.822310 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1422/2823] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:53:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:53:16.831648 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9927073-c007-45df-9e1e-1ab68d76ca15 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:53:16.833067 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-f9927073-c007-45df-9e1e-1ab68d76ca15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:53:16.890456 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9927073-c007-45df-9e1e-1ab68d76ca15 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:53:16.894070 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9198605-8b39-4e06-a275-35d9c95bc2e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:53:16.894709 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9198605-8b39-4e06-a275-35d9c95bc2e1 service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:53:16.894991 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9198605-8b39-4e06-a275-35d9c95bc2e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:53:16.895262 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9198605-8b39-4e06-a275-35d9c95bc2e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:53:16.904770 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9198605-8b39-4e06-a275-35d9c95bc2e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:53:16.906230 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9198605-8b39-4e06-a275-35d9c95bc2e1 service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:53:16.915578 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1402/2824] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:53:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:53:17.217111 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ddaca5-0ee0-4f7b-8956-fcfba1f2f4d3 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:53:17.218664 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-f8ddaca5-0ee0-4f7b-8956-fcfba1f2f4d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:53:17.232655 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ddaca5-0ee0-4f7b-8956-fcfba1f2f4d3 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:53:17.238486 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677418fa-fb9a-4bd0-9246-f4e7958f265d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:53:17.239394 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677418fa-fb9a-4bd0-9246-f4e7958f265d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:53:17.240966 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677418fa-fb9a-4bd0-9246-f4e7958f265d admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:53:17.241490 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677418fa-fb9a-4bd0-9246-f4e7958f265d admin admin] PATH_INFO: `/v3/projects/2e60fcf741034efca613038cd64a9f2a` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:53:17.242689 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677418fa-fb9a-4bd0-9246-f4e7958f265d admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e60fcf741034efca613038cd64a9f2a)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:53:17.244751 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677418fa-fb9a-4bd0-9246-f4e7958f265d admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:53:17.296244 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-677418fa-fb9a-4bd0-9246-f4e7958f265d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e60fcf741034efca613038cd64a9f2a', 'request_id': 'req-677418fa-fb9a-4bd0-9246-f4e7958f265d'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:53:17.305426 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-677418fa-fb9a-4bd0-9246-f4e7958f265d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e60fcf741034efca613038cd64a9f2a', 'request_id': 'req-677418fa-fb9a-4bd0-9246-f4e7958f265d'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:53:17.312100 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-677418fa-fb9a-4bd0-9246-f4e7958f265d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e60fcf741034efca613038cd64a9f2a', 'request_id': 'req-677418fa-fb9a-4bd0-9246-f4e7958f265d'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:53:17.323659 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-677418fa-fb9a-4bd0-9246-f4e7958f265d admin admin] The token cache is being invalidate because project 2e60fcf741034efca613038cd64a9f2a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:53:17.323866 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-677418fa-fb9a-4bd0-9246-f4e7958f265d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:53:17.324569 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-677418fa-fb9a-4bd0-9246-f4e7958f265d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:53:17.326115 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1423/2825] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jun 16 17:53:17 2022] DELETE /identity/v3/projects/2e60fcf741034efca613038cd64a9f2a => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:53:17.331244 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38031596-18b6-4849-abfa-dd3d3d67fd5d None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:53:17.331610 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38031596-18b6-4849-abfa-dd3d3d67fd5d None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:53:17.331964 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38031596-18b6-4849-abfa-dd3d3d67fd5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:53:17.349673 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-38031596-18b6-4849-abfa-dd3d3d67fd5d None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:53:17.350885 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-38031596-18b6-4849-abfa-dd3d3d67fd5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:53:17.369845 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:53:17.422910 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1403/2826] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:53:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:53:17.486910 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da52f80-3b25-4f0c-9ec8-3447df44a748 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:53:17.487156 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da52f80-3b25-4f0c-9ec8-3447df44a748 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:53:17.487439 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da52f80-3b25-4f0c-9ec8-3447df44a748 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:53:17.507309 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-5da52f80-3b25-4f0c-9ec8-3447df44a748 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:53:17.508547 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-5da52f80-3b25-4f0c-9ec8-3447df44a748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:53:17.521317 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38031596-18b6-4849-abfa-dd3d3d67fd5d None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:53:17.521826 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1424/2827] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:53:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:53:17.532760 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38031596-18b6-4849-abfa-dd3d3d67fd5d None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:53:17.538841 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4487e34c-ce35-4425-a5e3-2bb32f986929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:53:17.539605 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4487e34c-ce35-4425-a5e3-2bb32f986929 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:53:17.548035 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4487e34c-ce35-4425-a5e3-2bb32f986929 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:53:17.548448 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4487e34c-ce35-4425-a5e3-2bb32f986929 admin admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:53:17.549240 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4487e34c-ce35-4425-a5e3-2bb32f986929 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:53:17.549558 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4487e34c-ce35-4425-a5e3-2bb32f986929 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:53:17.551284 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4487e34c-ce35-4425-a5e3-2bb32f986929 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:53:17.569626 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1404/2828] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jun 16 17:53:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:53:17.574701 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da52f80-3b25-4f0c-9ec8-3447df44a748 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:53:17.584672 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da52f80-3b25-4f0c-9ec8-3447df44a748 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:53:17.589641 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52330fe-aa44-41f2-9003-1273db225352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:53:17.590394 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52330fe-aa44-41f2-9003-1273db225352 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:53:17.590744 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52330fe-aa44-41f2-9003-1273db225352 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:53:17.591088 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52330fe-aa44-41f2-9003-1273db225352 admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:53:17.591906 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52330fe-aa44-41f2-9003-1273db225352 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:53:17.593820 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52330fe-aa44-41f2-9003-1273db225352 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:53:17.622412 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1425/2829] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jun 16 17:53:17 2022] POST /identity/v3/projects => generated 371 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:53:17.625821 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4487e34c-ce35-4425-a5e3-2bb32f986929 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:53:17.636239 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4487e34c-ce35-4425-a5e3-2bb32f986929 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:53:17.695893 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0940a34-fdd5-4603-a8c4-805f2e3210b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:53:17.696584 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0940a34-fdd5-4603-a8c4-805f2e3210b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:53:17.696867 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0940a34-fdd5-4603-a8c4-805f2e3210b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:53:17.697137 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0940a34-fdd5-4603-a8c4-805f2e3210b6 admin admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:53:17.697820 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0940a34-fdd5-4603-a8c4-805f2e3210b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:53:17.699712 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0940a34-fdd5-4603-a8c4-805f2e3210b6 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:53:17.721655 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1405/2830] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jun 16 17:53:17 2022] POST /identity/v3/users => generated 286 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:53:17.722943 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52330fe-aa44-41f2-9003-1273db225352 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:53:17.734145 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52330fe-aa44-41f2-9003-1273db225352 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:53:17.738838 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7586ff55-de2f-481a-bace-ded08c1a4bc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:53:17.739497 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7586ff55-de2f-481a-bace-ded08c1a4bc1 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:53:17.740495 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7586ff55-de2f-481a-bace-ded08c1a4bc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:53:17.740788 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7586ff55-de2f-481a-bace-ded08c1a4bc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:53:17.741512 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7586ff55-de2f-481a-bace-ded08c1a4bc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:53:17.743104 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7586ff55-de2f-481a-bace-ded08c1a4bc1 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:53:17.753543 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1426/2831] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:53:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:53:17.757798 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0940a34-fdd5-4603-a8c4-805f2e3210b6 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:53:17.768024 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0940a34-fdd5-4603-a8c4-805f2e3210b6 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:53:17.773145 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fed24db-739f-49eb-bd4f-bddd899f1856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:53:17.773806 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fed24db-739f-49eb-bd4f-bddd899f1856 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:53:17.774092 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fed24db-739f-49eb-bd4f-bddd899f1856 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:53:17.774373 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fed24db-739f-49eb-bd4f-bddd899f1856 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:53:17.775102 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fed24db-739f-49eb-bd4f-bddd899f1856 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:53:17.776810 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fed24db-739f-49eb-bd4f-bddd899f1856 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:53:17.787364 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1406/2832] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:53:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:53:17.791716 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7586ff55-de2f-481a-bace-ded08c1a4bc1 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:53:17.801755 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7586ff55-de2f-481a-bace-ded08c1a4bc1 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:53:17.807269 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6c2c54-9935-4118-922d-1b1c90b82e2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:53:17.808146 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6c2c54-9935-4118-922d-1b1c90b82e2f admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:53:17.808466 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6c2c54-9935-4118-922d-1b1c90b82e2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:53:17.808747 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6c2c54-9935-4118-922d-1b1c90b82e2f admin admin] PATH_INFO: `/v3/projects/f2acc2c4ce1c48bca16bd4fec47ca650/users/1e8a07b42f3f47fcbc2eabd3143a4fab/roles/0be2075946f54c8485f1c275f746478a` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:53:17.824144 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6c2c54-9935-4118-922d-1b1c90b82e2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2acc2c4ce1c48bca16bd4fec47ca650, user_id=1e8a07b42f3f47fcbc2eabd3143a4fab, role_id=0be2075946f54c8485f1c275f746478a)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:53:17.826872 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6c2c54-9935-4118-922d-1b1c90b82e2f admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:53:17.840353 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1427/2833] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:53:17 2022] PUT /identity/v3/projects/f2acc2c4ce1c48bca16bd4fec47ca650/users/1e8a07b42f3f47fcbc2eabd3143a4fab/roles/0be2075946f54c8485f1c275f746478a => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:53:17.851170 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305946e5-7db2-48c0-8dcc-628f0f4c20b9 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:53:17.851551 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305946e5-7db2-48c0-8dcc-628f0f4c20b9 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:53:17.851898 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305946e5-7db2-48c0-8dcc-628f0f4c20b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:53:17.879350 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-305946e5-7db2-48c0-8dcc-628f0f4c20b9 None None] MFA Rules not processed for user `1e8a07b42f3f47fcbc2eabd3143a4fab`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:53:17.880622 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-305946e5-7db2-48c0-8dcc-628f0f4c20b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:53:17.944889 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1407/2834] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:53:17 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:53:17.956029 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6c2c54-9935-4118-922d-1b1c90b82e2f admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:53:17.957745 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-7c6c2c54-9935-4118-922d-1b1c90b82e2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:53:17.982175 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:53:18.020636 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6c2c54-9935-4118-922d-1b1c90b82e2f admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:53:18.024830 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b731f98-c7b5-481a-9b63-7b3fe6fd43c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24d5e2637b7043e08313909d5b06e329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:53:18.025465 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b731f98-c7b5-481a-9b63-7b3fe6fd43c8 service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:53:18.025747 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b731f98-c7b5-481a-9b63-7b3fe6fd43c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:53:18.026020 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b731f98-c7b5-481a-9b63-7b3fe6fd43c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:53:18.034482 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b731f98-c7b5-481a-9b63-7b3fe6fd43c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:53:18.035899 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b731f98-c7b5-481a-9b63-7b3fe6fd43c8 service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:53:18.046584 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1428/2835] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:53:17 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:53:18.071050 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305946e5-7db2-48c0-8dcc-628f0f4c20b9 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:53:18.090555 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:53:18.134868 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305946e5-7db2-48c0-8dcc-628f0f4c20b9 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:53:18.139800 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7d4680-b5bd-4d0c-8660-24a55bd72b33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:53:18.140695 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7d4680-b5bd-4d0c-8660-24a55bd72b33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:53:18.140979 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7d4680-b5bd-4d0c-8660-24a55bd72b33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:53:18.141256 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7d4680-b5bd-4d0c-8660-24a55bd72b33 admin admin] PATH_INFO: `/v3/users/1e8a07b42f3f47fcbc2eabd3143a4fab` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:53:18.142437 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7d4680-b5bd-4d0c-8660-24a55bd72b33 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e8a07b42f3f47fcbc2eabd3143a4fab)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:53:18.144692 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7d4680-b5bd-4d0c-8660-24a55bd72b33 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:53:18.190566 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-df7d4680-b5bd-4d0c-8660-24a55bd72b33 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e8a07b42f3f47fcbc2eabd3143a4fab'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:53:18.191224 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-df7d4680-b5bd-4d0c-8660-24a55bd72b33 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e8a07b42f3f47fcbc2eabd3143a4fab'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:53:18.198284 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-df7d4680-b5bd-4d0c-8660-24a55bd72b33 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e8a07b42f3f47fcbc2eabd3143a4fab'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:53:18.198920 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-df7d4680-b5bd-4d0c-8660-24a55bd72b33 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e8a07b42f3f47fcbc2eabd3143a4fab'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:53:18.211562 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-df7d4680-b5bd-4d0c-8660-24a55bd72b33 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e8a07b42f3f47fcbc2eabd3143a4fab'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:53:18.223632 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1408/2836] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jun 16 17:53:18 2022] DELETE /identity/v3/users/1e8a07b42f3f47fcbc2eabd3143a4fab => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:53:18.228883 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641fdc61-84fd-4c8a-8015-914bcb1f944f None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:53:18.229184 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641fdc61-84fd-4c8a-8015-914bcb1f944f None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:53:18.229468 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641fdc61-84fd-4c8a-8015-914bcb1f944f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:53:18.247024 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-641fdc61-84fd-4c8a-8015-914bcb1f944f None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:53:18.248237 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-641fdc61-84fd-4c8a-8015-914bcb1f944f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:53:18.306769 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1429/2837] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:53:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:53:18.316401 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7d4680-b5bd-4d0c-8660-24a55bd72b33 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:53:18.318541 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-df7d4680-b5bd-4d0c-8660-24a55bd72b33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:53:18.379331 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7d4680-b5bd-4d0c-8660-24a55bd72b33 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:53:18.383740 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914a7c2b-b5b2-4785-85ad-0f847579d05a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:53:18.384396 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914a7c2b-b5b2-4785-85ad-0f847579d05a service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:53:18.384679 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914a7c2b-b5b2-4785-85ad-0f847579d05a service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:53:18.384954 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914a7c2b-b5b2-4785-85ad-0f847579d05a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:53:18.394151 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914a7c2b-b5b2-4785-85ad-0f847579d05a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:53:18.395562 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914a7c2b-b5b2-4785-85ad-0f847579d05a service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:53:18.405495 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1409/2838] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:53:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:53:18.705488 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641fdc61-84fd-4c8a-8015-914bcb1f944f None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:53:18.706976 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-641fdc61-84fd-4c8a-8015-914bcb1f944f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:53:18.726801 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641fdc61-84fd-4c8a-8015-914bcb1f944f None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:53:18.742102 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdd5fc0-4542-460e-b5dd-c294a8e11401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:53:18.743454 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdd5fc0-4542-460e-b5dd-c294a8e11401 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:53:18.743744 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdd5fc0-4542-460e-b5dd-c294a8e11401 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:53:18.744029 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdd5fc0-4542-460e-b5dd-c294a8e11401 admin admin] PATH_INFO: `/v3/projects/f2acc2c4ce1c48bca16bd4fec47ca650` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:53:18.745332 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdd5fc0-4542-460e-b5dd-c294a8e11401 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2acc2c4ce1c48bca16bd4fec47ca650)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:53:18.747328 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdd5fc0-4542-460e-b5dd-c294a8e11401 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:53:18.816196 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-dbdd5fc0-4542-460e-b5dd-c294a8e11401 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2acc2c4ce1c48bca16bd4fec47ca650', 'request_id': 'req-dbdd5fc0-4542-460e-b5dd-c294a8e11401'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:53:18.824672 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-dbdd5fc0-4542-460e-b5dd-c294a8e11401 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2acc2c4ce1c48bca16bd4fec47ca650', 'request_id': 'req-dbdd5fc0-4542-460e-b5dd-c294a8e11401'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:53:18.831043 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-dbdd5fc0-4542-460e-b5dd-c294a8e11401 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2acc2c4ce1c48bca16bd4fec47ca650', 'request_id': 'req-dbdd5fc0-4542-460e-b5dd-c294a8e11401'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:53:18.842127 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-dbdd5fc0-4542-460e-b5dd-c294a8e11401 admin admin] The token cache is being invalidate because project f2acc2c4ce1c48bca16bd4fec47ca650 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:53:18.842411 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-dbdd5fc0-4542-460e-b5dd-c294a8e11401 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:53:18.843039 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-dbdd5fc0-4542-460e-b5dd-c294a8e11401 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:53:18.844617 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1430/2839] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jun 16 17:53:18 2022] DELETE /identity/v3/projects/f2acc2c4ce1c48bca16bd4fec47ca650 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:53:22.096352 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914a7c2b-b5b2-4785-85ad-0f847579d05a service neutron] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:53:22.097798 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-914a7c2b-b5b2-4785-85ad-0f847579d05a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:53:22.111517 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914a7c2b-b5b2-4785-85ad-0f847579d05a service neutron] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:53:22.150510 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:53:22.170631 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5783238f-f91c-424e-aec3-a1bfb377855b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:53:22.171505 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5783238f-f91c-424e-aec3-a1bfb377855b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:53:22.171786 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5783238f-f91c-424e-aec3-a1bfb377855b admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:53:22.172062 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5783238f-f91c-424e-aec3-a1bfb377855b admin admin] PATH_INFO: `/v3/users/50a264704a234809a7084f1117354cef` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:53:22.173274 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5783238f-f91c-424e-aec3-a1bfb377855b admin admin] RBAC: Authorizing `identity:delete_user(user_id=50a264704a234809a7084f1117354cef)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:53:22.175229 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5783238f-f91c-424e-aec3-a1bfb377855b admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:53:22.226295 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-5783238f-f91c-424e-aec3-a1bfb377855b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50a264704a234809a7084f1117354cef'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:53:22.227340 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-5783238f-f91c-424e-aec3-a1bfb377855b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50a264704a234809a7084f1117354cef'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:53:22.235777 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-5783238f-f91c-424e-aec3-a1bfb377855b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50a264704a234809a7084f1117354cef'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:53:22.236447 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-5783238f-f91c-424e-aec3-a1bfb377855b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50a264704a234809a7084f1117354cef'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:53:22.252851 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-5783238f-f91c-424e-aec3-a1bfb377855b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50a264704a234809a7084f1117354cef'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:53:22.266053 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1410/2840] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jun 16 17:53:22 2022] DELETE /identity/v3/users/50a264704a234809a7084f1117354cef => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:53:22.270433 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127caeb9-888c-434c-9372-95c9cc59647f None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:53:22.270934 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127caeb9-888c-434c-9372-95c9cc59647f None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:53:22.271051 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127caeb9-888c-434c-9372-95c9cc59647f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:53:22.289885 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-127caeb9-888c-434c-9372-95c9cc59647f None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:53:22.291299 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-127caeb9-888c-434c-9372-95c9cc59647f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:53:22.343440 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:53:22.373537 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1431/2841] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:53:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:53:22.383594 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5783238f-f91c-424e-aec3-a1bfb377855b admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:53:22.385083 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-5783238f-f91c-424e-aec3-a1bfb377855b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:53:22.445658 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5783238f-f91c-424e-aec3-a1bfb377855b admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:53:22.449259 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f04e2d6-0e55-49ca-9dcd-08d03dcd7990 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:53:22.449900 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f04e2d6-0e55-49ca-9dcd-08d03dcd7990 service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:53:22.450177 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f04e2d6-0e55-49ca-9dcd-08d03dcd7990 service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:53:22.450450 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f04e2d6-0e55-49ca-9dcd-08d03dcd7990 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:53:22.459440 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f04e2d6-0e55-49ca-9dcd-08d03dcd7990 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:53:22.461508 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f04e2d6-0e55-49ca-9dcd-08d03dcd7990 service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:53:22.473037 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1411/2842] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:53:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:53:23.006433 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127caeb9-888c-434c-9372-95c9cc59647f None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:53:23.008209 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-127caeb9-888c-434c-9372-95c9cc59647f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:53:23.020615 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127caeb9-888c-434c-9372-95c9cc59647f None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:53:23.025576 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0c1f5b-aa8c-4607-ab59-08946e648ded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:53:23.026467 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0c1f5b-aa8c-4607-ab59-08946e648ded admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:53:23.026782 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0c1f5b-aa8c-4607-ab59-08946e648ded admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:53:23.027031 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0c1f5b-aa8c-4607-ab59-08946e648ded admin admin] PATH_INFO: `/v3/projects/c6f92f56dd1344ca899912c704308955` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:53:23.028295 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0c1f5b-aa8c-4607-ab59-08946e648ded admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6f92f56dd1344ca899912c704308955)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:53:23.030386 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0c1f5b-aa8c-4607-ab59-08946e648ded admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:53:23.107580 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-9c0c1f5b-aa8c-4607-ab59-08946e648ded admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6f92f56dd1344ca899912c704308955', 'request_id': 'req-9c0c1f5b-aa8c-4607-ab59-08946e648ded'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:53:23.115879 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-9c0c1f5b-aa8c-4607-ab59-08946e648ded admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6f92f56dd1344ca899912c704308955', 'request_id': 'req-9c0c1f5b-aa8c-4607-ab59-08946e648ded'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:53:23.124205 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-9c0c1f5b-aa8c-4607-ab59-08946e648ded admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6f92f56dd1344ca899912c704308955', 'request_id': 'req-9c0c1f5b-aa8c-4607-ab59-08946e648ded'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:53:23.137434 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-9c0c1f5b-aa8c-4607-ab59-08946e648ded admin admin] The token cache is being invalidate because project c6f92f56dd1344ca899912c704308955 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:53:23.137987 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-9c0c1f5b-aa8c-4607-ab59-08946e648ded admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:53:23.138636 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-9c0c1f5b-aa8c-4607-ab59-08946e648ded admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:53:23.140262 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1432/2843] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jun 16 17:53:23 2022] DELETE /identity/v3/projects/c6f92f56dd1344ca899912c704308955 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:53:23.145860 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8109f6d-dc9e-413a-a773-5951845d9190 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:53:23.146802 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8109f6d-dc9e-413a-a773-5951845d9190 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:53:23.147448 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8109f6d-dc9e-413a-a773-5951845d9190 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:53:23.168237 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-d8109f6d-dc9e-413a-a773-5951845d9190 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:53:23.169712 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-d8109f6d-dc9e-413a-a773-5951845d9190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:53:23.233566 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1412/2844] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:53:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:53:23.243266 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed4bbee-20b4-44c8-8d9b-3da75356647b None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:53:23.243558 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed4bbee-20b4-44c8-8d9b-3da75356647b None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:53:23.243839 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed4bbee-20b4-44c8-8d9b-3da75356647b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:53:23.263871 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-5ed4bbee-20b4-44c8-8d9b-3da75356647b None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:53:23.265121 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-5ed4bbee-20b4-44c8-8d9b-3da75356647b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:53:23.273564 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1433/2845] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:53:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:53:23.277841 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8109f6d-dc9e-413a-a773-5951845d9190 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:53:23.282872 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:53:23.289548 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8109f6d-dc9e-413a-a773-5951845d9190 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:53:23.295348 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83aa8073-f9ef-4d3f-9e5e-a92cb3fd1f61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:53:23.296027 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aa8073-f9ef-4d3f-9e5e-a92cb3fd1f61 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:53:23.296308 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aa8073-f9ef-4d3f-9e5e-a92cb3fd1f61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:53:23.296634 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aa8073-f9ef-4d3f-9e5e-a92cb3fd1f61 admin admin] PATH_INFO: `/v3/domains` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:53:23.297360 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83aa8073-f9ef-4d3f-9e5e-a92cb3fd1f61 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80357) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:53:23.297665 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83aa8073-f9ef-4d3f-9e5e-a92cb3fd1f61 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:53:23.299328 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83aa8073-f9ef-4d3f-9e5e-a92cb3fd1f61 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:53:23.312291 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1413/2846] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jun 16 17:53:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:53:23.316812 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed4bbee-20b4-44c8-8d9b-3da75356647b None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:53:23.326804 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed4bbee-20b4-44c8-8d9b-3da75356647b None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:53:23.331492 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79288675-fc77-42f4-93eb-5ef091fda2dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:53:23.332145 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79288675-fc77-42f4-93eb-5ef091fda2dd admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:53:23.332457 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79288675-fc77-42f4-93eb-5ef091fda2dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:53:23.332736 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79288675-fc77-42f4-93eb-5ef091fda2dd admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:53:23.333428 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79288675-fc77-42f4-93eb-5ef091fda2dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:53:23.335599 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79288675-fc77-42f4-93eb-5ef091fda2dd admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:53:23.364735 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1434/2847] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jun 16 17:53:23 2022] POST /identity/v3/projects => generated 397 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:53:23.365759 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83aa8073-f9ef-4d3f-9e5e-a92cb3fd1f61 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:53:23.380128 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83aa8073-f9ef-4d3f-9e5e-a92cb3fd1f61 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:53:23.454382 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecf8b47-39f6-4504-be57-66e6224ef76c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:53:23.455062 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecf8b47-39f6-4504-be57-66e6224ef76c admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:53:23.455345 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecf8b47-39f6-4504-be57-66e6224ef76c admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:53:23.455622 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecf8b47-39f6-4504-be57-66e6224ef76c admin admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:53:23.456315 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecf8b47-39f6-4504-be57-66e6224ef76c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:53:23.458260 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecf8b47-39f6-4504-be57-66e6224ef76c admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:53:23.490295 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1414/2848] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jun 16 17:53:23 2022] POST /identity/v3/users => generated 299 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:53:23.494782 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79288675-fc77-42f4-93eb-5ef091fda2dd admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:53:23.506155 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79288675-fc77-42f4-93eb-5ef091fda2dd admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:53:23.511416 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cb1502-2f5e-4d7e-a91c-8c7b1bd807ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:53:23.512093 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cb1502-2f5e-4d7e-a91c-8c7b1bd807ac admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:53:23.512395 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cb1502-2f5e-4d7e-a91c-8c7b1bd807ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:53:23.513083 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cb1502-2f5e-4d7e-a91c-8c7b1bd807ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:53:23.513818 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cb1502-2f5e-4d7e-a91c-8c7b1bd807ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:53:23.515761 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cb1502-2f5e-4d7e-a91c-8c7b1bd807ac admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:53:23.528936 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1435/2849] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:53:23 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:53:23.532956 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecf8b47-39f6-4504-be57-66e6224ef76c admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:53:23.545068 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecf8b47-39f6-4504-be57-66e6224ef76c admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:53:23.559039 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c2f51b-2735-41a1-9a2d-b00d5b475980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:53:23.559718 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c2f51b-2735-41a1-9a2d-b00d5b475980 admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:53:23.559999 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c2f51b-2735-41a1-9a2d-b00d5b475980 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:53:23.560272 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c2f51b-2735-41a1-9a2d-b00d5b475980 admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:53:23.561032 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c2f51b-2735-41a1-9a2d-b00d5b475980 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:53:23.562624 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c2f51b-2735-41a1-9a2d-b00d5b475980 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:53:23.573704 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1415/2850] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:53:23 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:53:23.577986 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cb1502-2f5e-4d7e-a91c-8c7b1bd807ac admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:53:23.589137 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cb1502-2f5e-4d7e-a91c-8c7b1bd807ac admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:53:23.593959 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b980439e-5514-4d88-9e11-1991f6eb4c7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:53:23.594864 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b980439e-5514-4d88-9e11-1991f6eb4c7f admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:53:23.595150 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b980439e-5514-4d88-9e11-1991f6eb4c7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:53:23.595470 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b980439e-5514-4d88-9e11-1991f6eb4c7f admin admin] PATH_INFO: `/v3/projects/305c42d8c94d482b897f5687f4a89946/users/1003c4aed20c4e91a9c6949653b04c43/roles/0be2075946f54c8485f1c275f746478a` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:53:23.610554 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b980439e-5514-4d88-9e11-1991f6eb4c7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=305c42d8c94d482b897f5687f4a89946, user_id=1003c4aed20c4e91a9c6949653b04c43, role_id=0be2075946f54c8485f1c275f746478a)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:53:23.613320 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b980439e-5514-4d88-9e11-1991f6eb4c7f admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:53:23.627807 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1436/2851] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:53:23 2022] PUT /identity/v3/projects/305c42d8c94d482b897f5687f4a89946/users/1003c4aed20c4e91a9c6949653b04c43/roles/0be2075946f54c8485f1c275f746478a => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:53:23.639627 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a317aa5-278c-41bd-8545-3b5e92bd4fcd None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:53:23.639928 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a317aa5-278c-41bd-8545-3b5e92bd4fcd None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:53:23.640209 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a317aa5-278c-41bd-8545-3b5e92bd4fcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:53:23.668395 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-8a317aa5-278c-41bd-8545-3b5e92bd4fcd None None] MFA Rules not processed for user `1003c4aed20c4e91a9c6949653b04c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:53:23.669660 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-8a317aa5-278c-41bd-8545-3b5e92bd4fcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:53:23.735654 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1416/2852] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:53:23 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:53:23.747658 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b980439e-5514-4d88-9e11-1991f6eb4c7f admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:53:23.749517 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-b980439e-5514-4d88-9e11-1991f6eb4c7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:53:23.809335 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b980439e-5514-4d88-9e11-1991f6eb4c7f admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:53:23.813123 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb0be47-476b-4330-a8df-baab8e27a02e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24d5e2637b7043e08313909d5b06e329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:53:23.813768 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb0be47-476b-4330-a8df-baab8e27a02e service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:53:23.814055 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb0be47-476b-4330-a8df-baab8e27a02e service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:53:23.814337 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb0be47-476b-4330-a8df-baab8e27a02e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:53:23.823457 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb0be47-476b-4330-a8df-baab8e27a02e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:53:23.824945 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb0be47-476b-4330-a8df-baab8e27a02e service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:53:23.834809 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1437/2853] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:53:23 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:53:33.732432 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=71,keystone:INSERT=3 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:53:33.829770 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=86,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:53:35.450009 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a317aa5-278c-41bd-8545-3b5e92bd4fcd None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:53:35.451415 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-8a317aa5-278c-41bd-8545-3b5e92bd4fcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:53:35.512911 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a317aa5-278c-41bd-8545-3b5e92bd4fcd None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:53:35.516513 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5672415d-75bd-41f5-952a-04105c2028c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:53:35.517157 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5672415d-75bd-41f5-952a-04105c2028c3 service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:53:35.517440 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5672415d-75bd-41f5-952a-04105c2028c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:53:35.517716 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5672415d-75bd-41f5-952a-04105c2028c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:53:35.519484 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-5672415d-75bd-41f5-952a-04105c2028c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:53:35.582162 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5672415d-75bd-41f5-952a-04105c2028c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:53:35.583588 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5672415d-75bd-41f5-952a-04105c2028c3 service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:53:35.593394 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1417/2854] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:53:35 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:53:36.217620 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb0be47-476b-4330-a8df-baab8e27a02e service cinder] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:53:36.285204 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb0be47-476b-4330-a8df-baab8e27a02e service cinder] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:53:36.291194 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea5810f-d1ff-4133-bbd1-7ead2d906516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:53:36.292080 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea5810f-d1ff-4133-bbd1-7ead2d906516 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:53:36.292360 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea5810f-d1ff-4133-bbd1-7ead2d906516 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:53:36.292674 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea5810f-d1ff-4133-bbd1-7ead2d906516 admin admin] PATH_INFO: `/v3/users/1003c4aed20c4e91a9c6949653b04c43` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:53:36.293970 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea5810f-d1ff-4133-bbd1-7ead2d906516 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1003c4aed20c4e91a9c6949653b04c43)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:53:36.295914 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea5810f-d1ff-4133-bbd1-7ead2d906516 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:53:36.349078 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-5ea5810f-d1ff-4133-bbd1-7ead2d906516 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1003c4aed20c4e91a9c6949653b04c43'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:53:36.364171 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-5ea5810f-d1ff-4133-bbd1-7ead2d906516 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1003c4aed20c4e91a9c6949653b04c43'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:53:36.365067 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-5ea5810f-d1ff-4133-bbd1-7ead2d906516 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1003c4aed20c4e91a9c6949653b04c43'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:53:36.372107 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-5ea5810f-d1ff-4133-bbd1-7ead2d906516 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1003c4aed20c4e91a9c6949653b04c43'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:53:36.380255 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-5ea5810f-d1ff-4133-bbd1-7ead2d906516 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1003c4aed20c4e91a9c6949653b04c43'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:53:36.394160 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1438/2855] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jun 16 17:53:36 2022] DELETE /identity/v3/users/1003c4aed20c4e91a9c6949653b04c43 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:53:36.395839 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3214e9a2-6e50-48cd-ba8b-5505cef086c6 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:53:36.396132 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3214e9a2-6e50-48cd-ba8b-5505cef086c6 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:53:36.396475 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3214e9a2-6e50-48cd-ba8b-5505cef086c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:53:36.414580 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-3214e9a2-6e50-48cd-ba8b-5505cef086c6 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:53:36.415938 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-3214e9a2-6e50-48cd-ba8b-5505cef086c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:53:36.479168 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1418/2856] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:53:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:53:36.488914 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea5810f-d1ff-4133-bbd1-7ead2d906516 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:53:36.490394 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-5ea5810f-d1ff-4133-bbd1-7ead2d906516 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:53:36.550432 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea5810f-d1ff-4133-bbd1-7ead2d906516 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:53:36.556813 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb49dad-2dc5-4538-a51c-7465b5688a9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:53:36.557732 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb49dad-2dc5-4538-a51c-7465b5688a9a service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:53:36.558171 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb49dad-2dc5-4538-a51c-7465b5688a9a service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:53:36.558567 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb49dad-2dc5-4538-a51c-7465b5688a9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:53:36.570565 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb49dad-2dc5-4538-a51c-7465b5688a9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:53:36.572004 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb49dad-2dc5-4538-a51c-7465b5688a9a service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:53:36.582047 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1439/2857] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:53:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:53:37.018247 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3214e9a2-6e50-48cd-ba8b-5505cef086c6 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:53:37.019724 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-3214e9a2-6e50-48cd-ba8b-5505cef086c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:53:37.032669 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3214e9a2-6e50-48cd-ba8b-5505cef086c6 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:53:37.037484 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9603eb2-3443-4951-a771-df0e210808d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:53:37.038630 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9603eb2-3443-4951-a771-df0e210808d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:53:37.038927 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9603eb2-3443-4951-a771-df0e210808d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:53:37.039201 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9603eb2-3443-4951-a771-df0e210808d5 admin admin] PATH_INFO: `/v3/projects/305c42d8c94d482b897f5687f4a89946` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:53:37.040430 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9603eb2-3443-4951-a771-df0e210808d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=305c42d8c94d482b897f5687f4a89946)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:53:37.042359 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9603eb2-3443-4951-a771-df0e210808d5 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:53:37.105367 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:53:37.109442 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-a9603eb2-3443-4951-a771-df0e210808d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '305c42d8c94d482b897f5687f4a89946', 'request_id': 'req-a9603eb2-3443-4951-a771-df0e210808d5'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:53:37.124469 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-a9603eb2-3443-4951-a771-df0e210808d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '305c42d8c94d482b897f5687f4a89946', 'request_id': 'req-a9603eb2-3443-4951-a771-df0e210808d5'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:53:37.133176 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-a9603eb2-3443-4951-a771-df0e210808d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '305c42d8c94d482b897f5687f4a89946', 'request_id': 'req-a9603eb2-3443-4951-a771-df0e210808d5'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:53:37.150926 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-a9603eb2-3443-4951-a771-df0e210808d5 admin admin] The token cache is being invalidate because project 305c42d8c94d482b897f5687f4a89946 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80357) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:53:37.151317 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-a9603eb2-3443-4951-a771-df0e210808d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:53:37.152539 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-a9603eb2-3443-4951-a771-df0e210808d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:53:37.156035 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1419/2858] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jun 16 17:53:37 2022] DELETE /identity/v3/projects/305c42d8c94d482b897f5687f4a89946 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:53:37.161836 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dd5c50-5531-4660-aac6-0e00d106acae None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:53:37.162254 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dd5c50-5531-4660-aac6-0e00d106acae None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:53:37.163614 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dd5c50-5531-4660-aac6-0e00d106acae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:53:37.186557 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-68dd5c50-5531-4660-aac6-0e00d106acae None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:53:37.188197 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-68dd5c50-5531-4660-aac6-0e00d106acae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:53:37.213759 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:53:37.271042 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1440/2859] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:53:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:53:37.279176 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbc0180-262d-4c1e-b197-dc8059e815b9 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:53:37.279645 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbc0180-262d-4c1e-b197-dc8059e815b9 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:53:37.280109 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbc0180-262d-4c1e-b197-dc8059e815b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:53:37.301649 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-2dbc0180-262d-4c1e-b197-dc8059e815b9 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:53:37.303005 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-2dbc0180-262d-4c1e-b197-dc8059e815b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:53:37.311729 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1420/2860] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:53:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:53:37.316298 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dd5c50-5531-4660-aac6-0e00d106acae None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:53:37.327056 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dd5c50-5531-4660-aac6-0e00d106acae None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:53:37.332652 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235fae85-b7a2-40d8-8d57-220357198bea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:53:37.333417 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235fae85-b7a2-40d8-8d57-220357198bea admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:53:37.333777 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235fae85-b7a2-40d8-8d57-220357198bea admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:53:37.334130 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235fae85-b7a2-40d8-8d57-220357198bea admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:53:37.335079 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235fae85-b7a2-40d8-8d57-220357198bea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:53:37.335394 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235fae85-b7a2-40d8-8d57-220357198bea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:53:37.337167 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235fae85-b7a2-40d8-8d57-220357198bea admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:53:37.343989 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbc0180-262d-4c1e-b197-dc8059e815b9 None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:53:37.345842 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-2dbc0180-262d-4c1e-b197-dc8059e815b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:53:37.353217 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1441/2861] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jun 16 17:53:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:53:37.357235 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235fae85-b7a2-40d8-8d57-220357198bea admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:53:37.368330 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235fae85-b7a2-40d8-8d57-220357198bea admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:53:37.374163 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae2d5b8-edca-470e-b550-40c1ec28c8bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:53:37.374824 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae2d5b8-edca-470e-b550-40c1ec28c8bd admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:53:37.375107 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae2d5b8-edca-470e-b550-40c1ec28c8bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:53:37.375380 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae2d5b8-edca-470e-b550-40c1ec28c8bd admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:53:37.376066 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae2d5b8-edca-470e-b550-40c1ec28c8bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:53:37.378150 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae2d5b8-edca-470e-b550-40c1ec28c8bd admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:53:37.408753 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1442/2862] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jun 16 17:53:37 2022] POST /identity/v3/projects => generated 373 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:53:37.413249 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae2d5b8-edca-470e-b550-40c1ec28c8bd admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:53:37.431125 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbc0180-262d-4c1e-b197-dc8059e815b9 None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:53:37.487547 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae2d5b8-edca-470e-b550-40c1ec28c8bd admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:53:37.493293 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62fb0aa-ff9c-4f5c-8e21-82fbaea5e7e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:53:37.494037 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62fb0aa-ff9c-4f5c-8e21-82fbaea5e7e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:53:37.494503 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62fb0aa-ff9c-4f5c-8e21-82fbaea5e7e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:53:37.494851 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62fb0aa-ff9c-4f5c-8e21-82fbaea5e7e2 admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:53:37.495769 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62fb0aa-ff9c-4f5c-8e21-82fbaea5e7e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:53:37.497830 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62fb0aa-ff9c-4f5c-8e21-82fbaea5e7e2 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:53:37.504114 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb3fcdf-21bc-4cbc-92fb-50de8bbb739b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f86c16179a70474a817be9fda33ece9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:53:37.505013 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb3fcdf-21bc-4cbc-92fb-50de8bbb739b service nova] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:53:37.505238 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb3fcdf-21bc-4cbc-92fb-50de8bbb739b service nova] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:53:37.505518 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb3fcdf-21bc-4cbc-92fb-50de8bbb739b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:53:37.507650 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-3eb3fcdf-21bc-4cbc-92fb-50de8bbb739b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:53:37.521769 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb3fcdf-21bc-4cbc-92fb-50de8bbb739b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:53:37.523976 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb3fcdf-21bc-4cbc-92fb-50de8bbb739b service nova] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:53:37.526187 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1443/2863] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jun 16 17:53:37 2022] POST /identity/v3/users => generated 287 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:53:37.529999 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62fb0aa-ff9c-4f5c-8e21-82fbaea5e7e2 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:53:37.537343 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1421/2864] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:53:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:53:37.541114 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62fb0aa-ff9c-4f5c-8e21-82fbaea5e7e2 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:53:37.546443 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85822576-d31c-4dbf-8506-0818b9fc9b9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:53:37.547199 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85822576-d31c-4dbf-8506-0818b9fc9b9e admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:53:37.547552 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85822576-d31c-4dbf-8506-0818b9fc9b9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:53:37.547896 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85822576-d31c-4dbf-8506-0818b9fc9b9e admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:53:37.548764 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85822576-d31c-4dbf-8506-0818b9fc9b9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:53:37.550450 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85822576-d31c-4dbf-8506-0818b9fc9b9e admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:53:37.561953 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1444/2865] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:53:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:53:37.566097 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb3fcdf-21bc-4cbc-92fb-50de8bbb739b service nova] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:53:37.577725 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb3fcdf-21bc-4cbc-92fb-50de8bbb739b service nova] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:53:37.585402 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d2964c-714f-4a95-b2dd-87f74e6002be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:53:37.586110 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d2964c-714f-4a95-b2dd-87f74e6002be admin admin] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:53:37.586396 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d2964c-714f-4a95-b2dd-87f74e6002be admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:53:37.586674 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d2964c-714f-4a95-b2dd-87f74e6002be admin admin] PATH_INFO: `/v3/roles` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:53:37.587413 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d2964c-714f-4a95-b2dd-87f74e6002be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:53:37.589055 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d2964c-714f-4a95-b2dd-87f74e6002be admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:53:37.599953 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1422/2866] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:53:37 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:53:37.605841 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85822576-d31c-4dbf-8506-0818b9fc9b9e admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:53:37.616128 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85822576-d31c-4dbf-8506-0818b9fc9b9e admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:53:37.620873 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d6bfda-0a10-47f8-b29a-82cbfe3ce00c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:53:37.621872 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d6bfda-0a10-47f8-b29a-82cbfe3ce00c admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:53:37.622154 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d6bfda-0a10-47f8-b29a-82cbfe3ce00c admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:53:37.622426 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d6bfda-0a10-47f8-b29a-82cbfe3ce00c admin admin] PATH_INFO: `/v3/projects/10fb3764f6df42a7835b8458847cad0a/users/183c052ce8ea4899bf2c07bf63f67fd3/roles/0be2075946f54c8485f1c275f746478a` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:53:37.640730 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d6bfda-0a10-47f8-b29a-82cbfe3ce00c admin admin] RBAC: Authorizing `identity:create_grant(project_id=10fb3764f6df42a7835b8458847cad0a, user_id=183c052ce8ea4899bf2c07bf63f67fd3, role_id=0be2075946f54c8485f1c275f746478a)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:53:37.643598 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d6bfda-0a10-47f8-b29a-82cbfe3ce00c admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:53:37.660818 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1445/2867] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:53:37 2022] PUT /identity/v3/projects/10fb3764f6df42a7835b8458847cad0a/users/183c052ce8ea4899bf2c07bf63f67fd3/roles/0be2075946f54c8485f1c275f746478a => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:53:37.671423 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e7b7f0-c1f8-45da-9f8f-193bf3c42ee0 None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:53:37.671738 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e7b7f0-c1f8-45da-9f8f-193bf3c42ee0 None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:53:37.671996 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e7b7f0-c1f8-45da-9f8f-193bf3c42ee0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:53:37.694978 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-b1e7b7f0-c1f8-45da-9f8f-193bf3c42ee0 None None] MFA Rules not processed for user `12a220316b144048ad50f0affd87fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:53:37.696192 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-b1e7b7f0-c1f8-45da-9f8f-193bf3c42ee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:53:37.733673 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:53:37.768183 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1423/2868] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:53:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:53:37.778484 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d6bfda-0a10-47f8-b29a-82cbfe3ce00c admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:53:37.780473 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-22d6bfda-0a10-47f8-b29a-82cbfe3ce00c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:53:37.838668 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:53:37.856943 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d6bfda-0a10-47f8-b29a-82cbfe3ce00c admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:53:37.864479 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a71b953-c21c-4a4c-918d-47fe2a6d66a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:53:37.864479 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a71b953-c21c-4a4c-918d-47fe2a6d66a5 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:53:37.864479 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a71b953-c21c-4a4c-918d-47fe2a6d66a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:53:37.864479 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a71b953-c21c-4a4c-918d-47fe2a6d66a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:53:37.875574 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a71b953-c21c-4a4c-918d-47fe2a6d66a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:53:37.877174 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a71b953-c21c-4a4c-918d-47fe2a6d66a5 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:53:37.892621 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1446/2869] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:53:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 17:53:42.982168 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d30ae7a-52c7-4e07-96fa-737271c52baf None None] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:53:42.982726 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d30ae7a-52c7-4e07-96fa-737271c52baf None None] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:53:42.982828 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d30ae7a-52c7-4e07-96fa-737271c52baf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:53:43.014926 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.auth.core [None req-7d30ae7a-52c7-4e07-96fa-737271c52baf None None] MFA Rules not processed for user `183c052ce8ea4899bf2c07bf63f67fd3`. Rule list: `[]` (Enabled: `True`). {{(pid=80357) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:53:43.016741 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-7d30ae7a-52c7-4e07-96fa-737271c52baf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:53:43.084102 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1424/2870] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:53:42 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:53:43.088803 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a71b953-c21c-4a4c-918d-47fe2a6d66a5 service neutron] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:53:43.098623 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a71b953-c21c-4a4c-918d-47fe2a6d66a5 service neutron] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:53:43.104158 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb9b28c-f6ed-4c08-a7ba-f9b5659fbd5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:53:43.104930 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb9b28c-f6ed-4c08-a7ba-f9b5659fbd5a admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:53:43.105215 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb9b28c-f6ed-4c08-a7ba-f9b5659fbd5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:53:43.105493 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb9b28c-f6ed-4c08-a7ba-f9b5659fbd5a admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:53:43.106198 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb9b28c-f6ed-4c08-a7ba-f9b5659fbd5a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:53:43.108088 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb9b28c-f6ed-4c08-a7ba-f9b5659fbd5a admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:53:43.134666 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1447/2871] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jun 16 17:53:43 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:53:43.138922 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d30ae7a-52c7-4e07-96fa-737271c52baf None None] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:53:43.149709 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d30ae7a-52c7-4e07-96fa-737271c52baf None None] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:53:43.206375 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f727eaaf-4fb0-4755-947b-11c76e029360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:53:43.206959 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f727eaaf-4fb0-4755-947b-11c76e029360 admin admin] REQUEST_METHOD: `POST` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:53:43.207243 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f727eaaf-4fb0-4755-947b-11c76e029360 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:53:43.207519 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f727eaaf-4fb0-4755-947b-11c76e029360 admin admin] PATH_INFO: `/v3/users` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:53:43.208192 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f727eaaf-4fb0-4755-947b-11c76e029360 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:53:43.210375 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f727eaaf-4fb0-4755-947b-11c76e029360 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:53:43.230428 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1425/2872] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jun 16 17:53:43 2022] POST /identity/v3/users => generated 293 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:53:43.237577 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb9b28c-f6ed-4c08-a7ba-f9b5659fbd5a admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:53:43.249053 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb9b28c-f6ed-4c08-a7ba-f9b5659fbd5a admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:53:43.254177 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45b8552-c6d4-4883-8666-a5c8eb881c63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:53:43.254853 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45b8552-c6d4-4883-8666-a5c8eb881c63 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:53:43.255138 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45b8552-c6d4-4883-8666-a5c8eb881c63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:53:43.255416 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45b8552-c6d4-4883-8666-a5c8eb881c63 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:53:43.256143 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45b8552-c6d4-4883-8666-a5c8eb881c63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:53:43.258813 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45b8552-c6d4-4883-8666-a5c8eb881c63 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:53:43.274881 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1448/2873] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:53:43 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:53:43.275897 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f727eaaf-4fb0-4755-947b-11c76e029360 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:53:43.286330 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f727eaaf-4fb0-4755-947b-11c76e029360 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:53:43.291692 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eb0df1-d36d-4333-a61f-83830bfbd480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:53:43.292740 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eb0df1-d36d-4333-a61f-83830bfbd480 admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:53:43.293103 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eb0df1-d36d-4333-a61f-83830bfbd480 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:53:43.293448 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eb0df1-d36d-4333-a61f-83830bfbd480 admin admin] PATH_INFO: `/v3/projects/6143d2ad18614007aeac32aebaa20375/users/d353dcc0fd0c4d0e99ab0babb580c537/roles/3ca2dcbbd29d4a31a3ab78f2299bc239` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:53:43.310891 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3eb0df1-d36d-4333-a61f-83830bfbd480 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6143d2ad18614007aeac32aebaa20375, user_id=d353dcc0fd0c4d0e99ab0babb580c537, role_id=3ca2dcbbd29d4a31a3ab78f2299bc239)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:53:43.313684 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3eb0df1-d36d-4333-a61f-83830bfbd480 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:53:43.326862 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1426/2874] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:53:43 2022] PUT /identity/v3/projects/6143d2ad18614007aeac32aebaa20375/users/d353dcc0fd0c4d0e99ab0babb580c537/roles/3ca2dcbbd29d4a31a3ab78f2299bc239 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:53:43.330797 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45b8552-c6d4-4883-8666-a5c8eb881c63 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:53:43.342575 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45b8552-c6d4-4883-8666-a5c8eb881c63 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:53:43.403865 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a637634-feaa-4d68-90a3-ce59e70aede9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:53:43.404650 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a637634-feaa-4d68-90a3-ce59e70aede9 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:53:43.404936 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a637634-feaa-4d68-90a3-ce59e70aede9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:53:43.405217 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a637634-feaa-4d68-90a3-ce59e70aede9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:53:43.405984 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a637634-feaa-4d68-90a3-ce59e70aede9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:53:43.407592 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a637634-feaa-4d68-90a3-ce59e70aede9 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:53:43.418875 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1449/2875] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jun 16 17:53:43 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:53:43.423080 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eb0df1-d36d-4333-a61f-83830bfbd480 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:53:43.434065 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eb0df1-d36d-4333-a61f-83830bfbd480 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:53:43.438975 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58e19d4-ebb9-4bff-a7c8-171fd4f87db2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:53:43.439853 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58e19d4-ebb9-4bff-a7c8-171fd4f87db2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:53:43.440272 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58e19d4-ebb9-4bff-a7c8-171fd4f87db2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:53:43.440594 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58e19d4-ebb9-4bff-a7c8-171fd4f87db2 admin admin] PATH_INFO: `/v3/projects/6143d2ad18614007aeac32aebaa20375/users/d353dcc0fd0c4d0e99ab0babb580c537/roles/3ca2dcbbd29d4a31a3ab78f2299bc239` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:53:43.442552 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58e19d4-ebb9-4bff-a7c8-171fd4f87db2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6143d2ad18614007aeac32aebaa20375, user_id=d353dcc0fd0c4d0e99ab0babb580c537, role_id=3ca2dcbbd29d4a31a3ab78f2299bc239)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:53:43.445906 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58e19d4-ebb9-4bff-a7c8-171fd4f87db2 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:53:43.456297 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1427/2876] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jun 16 17:53:43 2022] PUT /identity/v3/projects/6143d2ad18614007aeac32aebaa20375/users/d353dcc0fd0c4d0e99ab0babb580c537/roles/3ca2dcbbd29d4a31a3ab78f2299bc239 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:53:48.217570 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d56dab-6315-4aa2-a9b8-7d10c530db4c None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:53:48.218049 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d56dab-6315-4aa2-a9b8-7d10c530db4c None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:53:48.218162 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d56dab-6315-4aa2-a9b8-7d10c530db4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:53:48.244650 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-86d56dab-6315-4aa2-a9b8-7d10c530db4c None None] MFA Rules not processed for user `d353dcc0fd0c4d0e99ab0babb580c537`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:53:48.245854 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-86d56dab-6315-4aa2-a9b8-7d10c530db4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:53:48.327150 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1450/2877] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:53:48 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:53:48.341533 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58e19d4-ebb9-4bff-a7c8-171fd4f87db2 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:53:48.370108 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:53:48.426298 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58e19d4-ebb9-4bff-a7c8-171fd4f87db2 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:53:48.437759 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ddfc05-9464-4da5-8017-a4b1c7d3bc1f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f86c16179a70474a817be9fda33ece9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:53:48.438431 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ddfc05-9464-4da5-8017-a4b1c7d3bc1f service nova] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:53:48.438714 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ddfc05-9464-4da5-8017-a4b1c7d3bc1f service nova] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:53:48.438990 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ddfc05-9464-4da5-8017-a4b1c7d3bc1f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:53:48.509262 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ddfc05-9464-4da5-8017-a4b1c7d3bc1f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:53:48.510950 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ddfc05-9464-4da5-8017-a4b1c7d3bc1f service nova] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:53:48.520937 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1428/2878] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:53:48 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:53:49.609515 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdd5dca-a59a-4597-8ca6-b81f2e648f75 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:53:49.610004 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdd5dca-a59a-4597-8ca6-b81f2e648f75 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:53:49.610103 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdd5dca-a59a-4597-8ca6-b81f2e648f75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:53:49.627874 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-1fdd5dca-a59a-4597-8ca6-b81f2e648f75 None None] MFA Rules not processed for user `183c052ce8ea4899bf2c07bf63f67fd3`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:53:49.629117 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-1fdd5dca-a59a-4597-8ca6-b81f2e648f75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:53:49.636066 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1451/2879] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:53:49 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:53:58.516423 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=46 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:53:59.622639 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=87,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:54:06.665338 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ddfc05-9464-4da5-8017-a4b1c7d3bc1f service nova] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:54:06.666806 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-e5ddfc05-9464-4da5-8017-a4b1c7d3bc1f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:54:06.732852 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ddfc05-9464-4da5-8017-a4b1c7d3bc1f service nova] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:54:06.737557 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5af249-b5c3-4fef-82ba-f0e7813f0890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:54:06.738424 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5af249-b5c3-4fef-82ba-f0e7813f0890 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:54:06.738715 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5af249-b5c3-4fef-82ba-f0e7813f0890 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:54:06.738993 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5af249-b5c3-4fef-82ba-f0e7813f0890 admin admin] PATH_INFO: `/v3/users/0824d8df380e477b85690814b0a2a9e3` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:54:06.740215 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5af249-b5c3-4fef-82ba-f0e7813f0890 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0824d8df380e477b85690814b0a2a9e3)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:54:06.742866 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5af249-b5c3-4fef-82ba-f0e7813f0890 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:54:06.794812 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-cb5af249-b5c3-4fef-82ba-f0e7813f0890 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0824d8df380e477b85690814b0a2a9e3'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:54:06.795479 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-cb5af249-b5c3-4fef-82ba-f0e7813f0890 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0824d8df380e477b85690814b0a2a9e3'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:54:06.804077 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-cb5af249-b5c3-4fef-82ba-f0e7813f0890 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0824d8df380e477b85690814b0a2a9e3'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:54:06.805681 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-cb5af249-b5c3-4fef-82ba-f0e7813f0890 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0824d8df380e477b85690814b0a2a9e3'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:54:06.820708 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-cb5af249-b5c3-4fef-82ba-f0e7813f0890 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0824d8df380e477b85690814b0a2a9e3'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:54:06.834284 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1429/2880] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jun 16 17:54:06 2022] DELETE /identity/v3/users/0824d8df380e477b85690814b0a2a9e3 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:54:07.022084 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdd5dca-a59a-4597-8ca6-b81f2e648f75 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:54:07.035710 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdd5dca-a59a-4597-8ca6-b81f2e648f75 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:54:07.095544 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51651fa0-9ff5-454f-b0be-0976edeef78b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:54:07.096458 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51651fa0-9ff5-454f-b0be-0976edeef78b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:54:07.096737 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51651fa0-9ff5-454f-b0be-0976edeef78b admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:54:07.097091 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51651fa0-9ff5-454f-b0be-0976edeef78b admin admin] PATH_INFO: `/v3/projects/55704b6435104c8e958ae6c5bb202f24` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:54:07.098495 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51651fa0-9ff5-454f-b0be-0976edeef78b admin admin] RBAC: Authorizing `identity:delete_project(project_id=55704b6435104c8e958ae6c5bb202f24)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:54:07.100442 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51651fa0-9ff5-454f-b0be-0976edeef78b admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:54:07.148266 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-51651fa0-9ff5-454f-b0be-0976edeef78b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55704b6435104c8e958ae6c5bb202f24', 'request_id': 'req-51651fa0-9ff5-454f-b0be-0976edeef78b'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:54:07.155361 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-51651fa0-9ff5-454f-b0be-0976edeef78b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55704b6435104c8e958ae6c5bb202f24', 'request_id': 'req-51651fa0-9ff5-454f-b0be-0976edeef78b'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:54:07.161814 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-51651fa0-9ff5-454f-b0be-0976edeef78b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55704b6435104c8e958ae6c5bb202f24', 'request_id': 'req-51651fa0-9ff5-454f-b0be-0976edeef78b'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:54:07.173205 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-51651fa0-9ff5-454f-b0be-0976edeef78b admin admin] The token cache is being invalidate because project 55704b6435104c8e958ae6c5bb202f24 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:54:07.173493 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-51651fa0-9ff5-454f-b0be-0976edeef78b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:54:07.174124 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-51651fa0-9ff5-454f-b0be-0976edeef78b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:54:07.175768 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1452/2881] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jun 16 17:54:07 2022] DELETE /identity/v3/projects/55704b6435104c8e958ae6c5bb202f24 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:54:08.060248 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5af249-b5c3-4fef-82ba-f0e7813f0890 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:54:08.061715 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-cb5af249-b5c3-4fef-82ba-f0e7813f0890 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:54:08.123560 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5af249-b5c3-4fef-82ba-f0e7813f0890 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:54:08.155834 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:54:08.174802 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee151414-dc60-49ba-ac38-e1c7e939c209 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24d5e2637b7043e08313909d5b06e329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:54:08.175442 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee151414-dc60-49ba-ac38-e1c7e939c209 service cinder] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:54:08.175722 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee151414-dc60-49ba-ac38-e1c7e939c209 service cinder] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:54:08.175993 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee151414-dc60-49ba-ac38-e1c7e939c209 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:54:08.177759 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-ee151414-dc60-49ba-ac38-e1c7e939c209 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:54:08.246801 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee151414-dc60-49ba-ac38-e1c7e939c209 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:54:08.248228 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee151414-dc60-49ba-ac38-e1c7e939c209 service cinder] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:54:08.259360 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1430/2882] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:54:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:54:09.002920 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51651fa0-9ff5-454f-b0be-0976edeef78b admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:54:09.004363 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-51651fa0-9ff5-454f-b0be-0976edeef78b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:54:09.066654 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51651fa0-9ff5-454f-b0be-0976edeef78b admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:54:09.072238 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b428a50-49f9-451d-92a9-8a89654d3f49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:54:09.072948 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b428a50-49f9-451d-92a9-8a89654d3f49 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:54:09.073204 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b428a50-49f9-451d-92a9-8a89654d3f49 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:54:09.073482 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b428a50-49f9-451d-92a9-8a89654d3f49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:54:09.075295 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-9b428a50-49f9-451d-92a9-8a89654d3f49 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:54:09.093941 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b428a50-49f9-451d-92a9-8a89654d3f49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:54:09.095493 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b428a50-49f9-451d-92a9-8a89654d3f49 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:54:09.109053 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1453/2883] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:54:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 17:54:18.254240 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=38 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:54:19.103604 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=77,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:54:25.843975 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee151414-dc60-49ba-ac38-e1c7e939c209 service cinder] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:54:25.854370 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee151414-dc60-49ba-ac38-e1c7e939c209 service cinder] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:54:25.858006 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7e3c80-439c-41fd-bdd1-652ba38e54af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:54:25.858640 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7e3c80-439c-41fd-bdd1-652ba38e54af service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:54:25.858919 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7e3c80-439c-41fd-bdd1-652ba38e54af service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:54:25.859189 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7e3c80-439c-41fd-bdd1-652ba38e54af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:54:25.860947 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-9e7e3c80-439c-41fd-bdd1-652ba38e54af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:54:25.924043 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7e3c80-439c-41fd-bdd1-652ba38e54af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:54:25.925509 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7e3c80-439c-41fd-bdd1-652ba38e54af service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:54:25.935737 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1431/2884] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:54:25 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:54:35.931418 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=30 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:54:56.113961 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b428a50-49f9-451d-92a9-8a89654d3f49 service neutron] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:54:56.123335 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b428a50-49f9-451d-92a9-8a89654d3f49 service neutron] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:54:56.126968 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052956f1-da77-47a2-80d2-55503e8c29b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:54:56.127601 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052956f1-da77-47a2-80d2-55503e8c29b9 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:54:56.127883 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052956f1-da77-47a2-80d2-55503e8c29b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:54:56.128159 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052956f1-da77-47a2-80d2-55503e8c29b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:54:56.129941 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-052956f1-da77-47a2-80d2-55503e8c29b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:54:56.140657 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052956f1-da77-47a2-80d2-55503e8c29b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:54:56.142015 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052956f1-da77-47a2-80d2-55503e8c29b9 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:54:56.150057 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1454/2885] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:54:56 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:55:06.146417 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=6 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:55:48.677693 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7e3c80-439c-41fd-bdd1-652ba38e54af service neutron] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:55:48.679367 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-9e7e3c80-439c-41fd-bdd1-652ba38e54af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:55:48.697416 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7e3c80-439c-41fd-bdd1-652ba38e54af service neutron] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:55:48.703383 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd3a5ff-5716-4f4a-b1cd-a5731621f264 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f86c16179a70474a817be9fda33ece9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:55:48.704044 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd3a5ff-5716-4f4a-b1cd-a5731621f264 service nova] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:55:48.704326 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd3a5ff-5716-4f4a-b1cd-a5731621f264 service nova] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:55:48.704629 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd3a5ff-5716-4f4a-b1cd-a5731621f264 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:55:48.706490 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-ccd3a5ff-5716-4f4a-b1cd-a5731621f264 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:55:48.719928 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd3a5ff-5716-4f4a-b1cd-a5731621f264 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:55:48.721638 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd3a5ff-5716-4f4a-b1cd-a5731621f264 service nova] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:55:48.738258 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1432/2886] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:55:48 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:55:50.465185 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9317c9c1-0a8c-4a15-8b4b-44b15fad4753 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:55:50.465701 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9317c9c1-0a8c-4a15-8b4b-44b15fad4753 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:55:50.465823 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9317c9c1-0a8c-4a15-8b4b-44b15fad4753 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:55:50.484734 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-9317c9c1-0a8c-4a15-8b4b-44b15fad4753 None None] MFA Rules not processed for user `31e04db1b0f6444e9a884a0e58ba38da`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:55:50.485935 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-9317c9c1-0a8c-4a15-8b4b-44b15fad4753 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:55:50.540284 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1455/2887] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:55:50 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:55:51.326938 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd3a5ff-5716-4f4a-b1cd-a5731621f264 service nova] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:55:51.328544 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-ccd3a5ff-5716-4f4a-b1cd-a5731621f264 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:55:51.400522 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd3a5ff-5716-4f4a-b1cd-a5731621f264 service nova] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:55:51.405535 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194ac862-4960-42c7-b8ad-5a2d4f09a16b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0570d8d9508d4546ae897276f7d181d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:55:51.406180 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194ac862-4960-42c7-b8ad-5a2d4f09a16b service placement] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:55:51.406466 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194ac862-4960-42c7-b8ad-5a2d4f09a16b service placement] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:55:51.406739 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194ac862-4960-42c7-b8ad-5a2d4f09a16b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:55:51.408528 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-194ac862-4960-42c7-b8ad-5a2d4f09a16b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:55:51.420227 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194ac862-4960-42c7-b8ad-5a2d4f09a16b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:55:51.421830 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194ac862-4960-42c7-b8ad-5a2d4f09a16b service placement] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:55:51.432666 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1433/2888] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:55:51 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:56:00.533020 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=25 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:56:01.427473 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=38 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:56:07.406655 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9317c9c1-0a8c-4a15-8b4b-44b15fad4753 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:56:07.408186 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-9317c9c1-0a8c-4a15-8b4b-44b15fad4753 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:56:07.477751 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9317c9c1-0a8c-4a15-8b4b-44b15fad4753 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:56:07.484281 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0f5b2a-1948-48b1-8c7f-843ffa69d2e6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f86c16179a70474a817be9fda33ece9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:56:07.485062 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0f5b2a-1948-48b1-8c7f-843ffa69d2e6 service nova] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:56:07.485354 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0f5b2a-1948-48b1-8c7f-843ffa69d2e6 service nova] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:56:07.485638 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0f5b2a-1948-48b1-8c7f-843ffa69d2e6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:56:07.487432 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-6d0f5b2a-1948-48b1-8c7f-843ffa69d2e6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:56:07.500936 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0f5b2a-1948-48b1-8c7f-843ffa69d2e6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:56:07.502443 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0f5b2a-1948-48b1-8c7f-843ffa69d2e6 service nova] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:56:07.514059 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1456/2889] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:56:07 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:56:17.509286 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=28 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:56:38.968329 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194ac862-4960-42c7-b8ad-5a2d4f09a16b service placement] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:56:39.017424 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194ac862-4960-42c7-b8ad-5a2d4f09a16b service placement] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:56:39.021475 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b7a57e-ed3b-4553-bae5-aa89de4edf3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:56:39.022066 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b7a57e-ed3b-4553-bae5-aa89de4edf3c service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:56:39.022350 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b7a57e-ed3b-4553-bae5-aa89de4edf3c service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:56:39.022627 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b7a57e-ed3b-4553-bae5-aa89de4edf3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:56:39.024672 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-29b7a57e-ed3b-4553-bae5-aa89de4edf3c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:56:39.052846 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b7a57e-ed3b-4553-bae5-aa89de4edf3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:56:39.055415 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b7a57e-ed3b-4553-bae5-aa89de4edf3c service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:56:39.065687 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1434/2890] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:56:38 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:56:39.160324 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0f5b2a-1948-48b1-8c7f-843ffa69d2e6 service nova] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:56:39.161743 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-6d0f5b2a-1948-48b1-8c7f-843ffa69d2e6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:56:39.173292 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0f5b2a-1948-48b1-8c7f-843ffa69d2e6 service nova] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:56:39.176866 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8528b5ca-df41-4ba6-b705-d795426918b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:56:39.177493 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8528b5ca-df41-4ba6-b705-d795426918b3 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:56:39.177774 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8528b5ca-df41-4ba6-b705-d795426918b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:56:39.178047 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8528b5ca-df41-4ba6-b705-d795426918b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:56:39.179748 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-8528b5ca-df41-4ba6-b705-d795426918b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:56:39.201246 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8528b5ca-df41-4ba6-b705-d795426918b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:56:39.202705 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8528b5ca-df41-4ba6-b705-d795426918b3 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:56:39.214293 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1457/2891] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:56:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:56:49.061459 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=23 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:56:49.208763 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=9 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:57:40.255728 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b7a57e-ed3b-4553-bae5-aa89de4edf3c service neutron] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:57:40.257197 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-29b7a57e-ed3b-4553-bae5-aa89de4edf3c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:57:40.269890 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b7a57e-ed3b-4553-bae5-aa89de4edf3c service neutron] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:57:40.273546 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b7beec-09a9-4816-aeba-2d8a3e71143e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24d5e2637b7043e08313909d5b06e329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:57:40.274187 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b7beec-09a9-4816-aeba-2d8a3e71143e service cinder] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:57:40.274473 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b7beec-09a9-4816-aeba-2d8a3e71143e service cinder] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:57:40.274750 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b7beec-09a9-4816-aeba-2d8a3e71143e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:57:40.276506 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-d2b7beec-09a9-4816-aeba-2d8a3e71143e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:57:40.287345 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b7beec-09a9-4816-aeba-2d8a3e71143e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:57:40.288809 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b7beec-09a9-4816-aeba-2d8a3e71143e service cinder] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:57:40.298123 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1435/2892] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:57:40 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:57:48.148294 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1925eacb-fe57-4c68-a489-8773621ec2cc None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:57:48.148843 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1925eacb-fe57-4c68-a489-8773621ec2cc None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:57:48.148843 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1925eacb-fe57-4c68-a489-8773621ec2cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:57:48.166468 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-1925eacb-fe57-4c68-a489-8773621ec2cc None None] MFA Rules not processed for user `31e04db1b0f6444e9a884a0e58ba38da`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:57:48.167665 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-1925eacb-fe57-4c68-a489-8773621ec2cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:57:48.175844 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1458/2893] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jun 16 17:57:48 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:57:48.501055 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b7beec-09a9-4816-aeba-2d8a3e71143e service cinder] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:57:48.510638 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b7beec-09a9-4816-aeba-2d8a3e71143e service cinder] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:57:48.514668 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bf5a3e-83d7-4fe5-b35e-35b1111660b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:57:48.515302 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bf5a3e-83d7-4fe5-b35e-35b1111660b7 service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:57:48.515588 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bf5a3e-83d7-4fe5-b35e-35b1111660b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:57:48.515865 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bf5a3e-83d7-4fe5-b35e-35b1111660b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:57:48.517629 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-59bf5a3e-83d7-4fe5-b35e-35b1111660b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:57:48.527076 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bf5a3e-83d7-4fe5-b35e-35b1111660b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:57:48.529214 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bf5a3e-83d7-4fe5-b35e-35b1111660b7 service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:57:48.537314 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1436/2894] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:57:48 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:57:49.518300 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1925eacb-fe57-4c68-a489-8773621ec2cc None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:57:49.544113 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1925eacb-fe57-4c68-a489-8773621ec2cc None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:57:49.549366 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6c8aa3-73e3-4368-847b-d712cc9b0fb7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0570d8d9508d4546ae897276f7d181d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:57:49.550017 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6c8aa3-73e3-4368-847b-d712cc9b0fb7 service placement] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:57:49.550298 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6c8aa3-73e3-4368-847b-d712cc9b0fb7 service placement] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:57:49.550579 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6c8aa3-73e3-4368-847b-d712cc9b0fb7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:57:49.552402 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-cf6c8aa3-73e3-4368-847b-d712cc9b0fb7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:57:49.580630 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6c8aa3-73e3-4368-847b-d712cc9b0fb7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:57:49.582455 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6c8aa3-73e3-4368-847b-d712cc9b0fb7 service placement] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:57:49.596008 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1459/2895] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:57:49 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:57:58.533741 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=12 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:57:59.591080 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=18 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:58:16.182086 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bf5a3e-83d7-4fe5-b35e-35b1111660b7 service neutron] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:58:16.184162 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-59bf5a3e-83d7-4fe5-b35e-35b1111660b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:58:16.199869 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bf5a3e-83d7-4fe5-b35e-35b1111660b7 service neutron] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:58:16.205530 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0859b00-441c-4e6a-8e4e-6a0ef4ba6a6e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0570d8d9508d4546ae897276f7d181d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:58:16.206180 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0859b00-441c-4e6a-8e4e-6a0ef4ba6a6e service placement] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:58:16.206466 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0859b00-441c-4e6a-8e4e-6a0ef4ba6a6e service placement] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:58:16.206744 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0859b00-441c-4e6a-8e4e-6a0ef4ba6a6e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:58:16.208778 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-f0859b00-441c-4e6a-8e4e-6a0ef4ba6a6e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:58:16.222082 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0859b00-441c-4e6a-8e4e-6a0ef4ba6a6e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:58:16.224008 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0859b00-441c-4e6a-8e4e-6a0ef4ba6a6e service placement] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:58:16.235943 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1437/2896] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:58:16 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:58:19.448088 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6c8aa3-73e3-4368-847b-d712cc9b0fb7 service placement] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:58:19.450979 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-cf6c8aa3-73e3-4368-847b-d712cc9b0fb7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:58:19.569706 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6c8aa3-73e3-4368-847b-d712cc9b0fb7 service placement] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:58:19.574773 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ba1770-b562-453c-b224-7f7b0367e6e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:58:19.575635 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ba1770-b562-453c-b224-7f7b0367e6e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:58:19.575919 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ba1770-b562-453c-b224-7f7b0367e6e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:58:19.576193 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ba1770-b562-453c-b224-7f7b0367e6e6 admin admin] PATH_INFO: `/v3/users/183c052ce8ea4899bf2c07bf63f67fd3` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:58:19.577405 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ba1770-b562-453c-b224-7f7b0367e6e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=183c052ce8ea4899bf2c07bf63f67fd3)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:58:19.579338 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ba1770-b562-453c-b224-7f7b0367e6e6 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:58:19.626448 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-a5ba1770-b562-453c-b224-7f7b0367e6e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '183c052ce8ea4899bf2c07bf63f67fd3'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:58:19.639059 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-a5ba1770-b562-453c-b224-7f7b0367e6e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '183c052ce8ea4899bf2c07bf63f67fd3'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:58:19.639768 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-a5ba1770-b562-453c-b224-7f7b0367e6e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '183c052ce8ea4899bf2c07bf63f67fd3'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:58:19.649504 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-a5ba1770-b562-453c-b224-7f7b0367e6e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '183c052ce8ea4899bf2c07bf63f67fd3'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:58:19.658376 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-a5ba1770-b562-453c-b224-7f7b0367e6e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '183c052ce8ea4899bf2c07bf63f67fd3'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:58:19.664357 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1460/2897] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jun 16 17:58:19 2022] DELETE /identity/v3/users/183c052ce8ea4899bf2c07bf63f67fd3 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:58:19.845430 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0859b00-441c-4e6a-8e4e-6a0ef4ba6a6e service placement] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:58:19.857823 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0859b00-441c-4e6a-8e4e-6a0ef4ba6a6e service placement] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:58:19.922180 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a120deb2-a9fa-4026-aa61-a9e583a95df7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:58:19.923037 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a120deb2-a9fa-4026-aa61-a9e583a95df7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:58:19.923317 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a120deb2-a9fa-4026-aa61-a9e583a95df7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:58:19.923591 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a120deb2-a9fa-4026-aa61-a9e583a95df7 admin admin] PATH_INFO: `/v3/projects/10fb3764f6df42a7835b8458847cad0a` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:58:19.924810 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a120deb2-a9fa-4026-aa61-a9e583a95df7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=10fb3764f6df42a7835b8458847cad0a)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:58:19.926702 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a120deb2-a9fa-4026-aa61-a9e583a95df7 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:58:19.971965 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-a120deb2-a9fa-4026-aa61-a9e583a95df7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10fb3764f6df42a7835b8458847cad0a', 'request_id': 'req-a120deb2-a9fa-4026-aa61-a9e583a95df7'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:58:19.979589 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-a120deb2-a9fa-4026-aa61-a9e583a95df7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10fb3764f6df42a7835b8458847cad0a', 'request_id': 'req-a120deb2-a9fa-4026-aa61-a9e583a95df7'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:58:19.987091 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-a120deb2-a9fa-4026-aa61-a9e583a95df7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10fb3764f6df42a7835b8458847cad0a', 'request_id': 'req-a120deb2-a9fa-4026-aa61-a9e583a95df7'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:58:20.007942 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-a120deb2-a9fa-4026-aa61-a9e583a95df7 admin admin] The token cache is being invalidate because project 10fb3764f6df42a7835b8458847cad0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80357) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:58:20.008247 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-a120deb2-a9fa-4026-aa61-a9e583a95df7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:58:20.008935 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-a120deb2-a9fa-4026-aa61-a9e583a95df7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:58:20.010480 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1438/2898] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jun 16 17:58:19 2022] DELETE /identity/v3/projects/10fb3764f6df42a7835b8458847cad0a => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:58:20.015787 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ba1770-b562-453c-b224-7f7b0367e6e6 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:58:20.017265 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-a5ba1770-b562-453c-b224-7f7b0367e6e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:58:20.030351 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ba1770-b562-453c-b224-7f7b0367e6e6 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:58:20.078906 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:58:20.091076 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bcce6f-37b5-486b-9011-787391e16adb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:58:20.091948 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bcce6f-37b5-486b-9011-787391e16adb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:58:20.092232 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bcce6f-37b5-486b-9011-787391e16adb admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:58:20.092550 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bcce6f-37b5-486b-9011-787391e16adb admin admin] PATH_INFO: `/v3/users/d353dcc0fd0c4d0e99ab0babb580c537` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:58:20.093733 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bcce6f-37b5-486b-9011-787391e16adb admin admin] RBAC: Authorizing `identity:delete_user(user_id=d353dcc0fd0c4d0e99ab0babb580c537)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:58:20.095633 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bcce6f-37b5-486b-9011-787391e16adb admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:58:20.143415 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-55bcce6f-37b5-486b-9011-787391e16adb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd353dcc0fd0c4d0e99ab0babb580c537'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:58:20.156155 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-55bcce6f-37b5-486b-9011-787391e16adb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd353dcc0fd0c4d0e99ab0babb580c537'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:58:20.156887 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-55bcce6f-37b5-486b-9011-787391e16adb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd353dcc0fd0c4d0e99ab0babb580c537'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:58:20.163898 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-55bcce6f-37b5-486b-9011-787391e16adb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd353dcc0fd0c4d0e99ab0babb580c537'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:58:20.172157 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-55bcce6f-37b5-486b-9011-787391e16adb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd353dcc0fd0c4d0e99ab0babb580c537'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:58:20.178117 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1461/2899] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jun 16 17:58:20 2022] DELETE /identity/v3/users/d353dcc0fd0c4d0e99ab0babb580c537 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:58:20.355341 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a120deb2-a9fa-4026-aa61-a9e583a95df7 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:58:20.356859 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-a120deb2-a9fa-4026-aa61-a9e583a95df7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:58:20.369511 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a120deb2-a9fa-4026-aa61-a9e583a95df7 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:58:20.428825 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa330e56-1b48-43b4-b487-11b3f2b19734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:58:20.429615 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa330e56-1b48-43b4-b487-11b3f2b19734 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:58:20.429899 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa330e56-1b48-43b4-b487-11b3f2b19734 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:58:20.430179 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa330e56-1b48-43b4-b487-11b3f2b19734 admin admin] PATH_INFO: `/v3/projects/6143d2ad18614007aeac32aebaa20375` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:58:20.431354 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa330e56-1b48-43b4-b487-11b3f2b19734 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6143d2ad18614007aeac32aebaa20375)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:58:20.433554 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa330e56-1b48-43b4-b487-11b3f2b19734 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:58:20.454945 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:58:20.487734 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-aa330e56-1b48-43b4-b487-11b3f2b19734 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6143d2ad18614007aeac32aebaa20375', 'request_id': 'req-aa330e56-1b48-43b4-b487-11b3f2b19734'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:58:20.498128 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-aa330e56-1b48-43b4-b487-11b3f2b19734 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6143d2ad18614007aeac32aebaa20375', 'request_id': 'req-aa330e56-1b48-43b4-b487-11b3f2b19734'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:58:20.504954 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-aa330e56-1b48-43b4-b487-11b3f2b19734 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6143d2ad18614007aeac32aebaa20375', 'request_id': 'req-aa330e56-1b48-43b4-b487-11b3f2b19734'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:58:20.517665 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-aa330e56-1b48-43b4-b487-11b3f2b19734 admin admin] The token cache is being invalidate because project 6143d2ad18614007aeac32aebaa20375 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80357) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:58:20.517960 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-aa330e56-1b48-43b4-b487-11b3f2b19734 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:58:20.518653 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-aa330e56-1b48-43b4-b487-11b3f2b19734 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:58:20.520217 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1439/2900] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jun 16 17:58:20 2022] DELETE /identity/v3/projects/6143d2ad18614007aeac32aebaa20375 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:58:30.174529 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=30,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:58:30.514543 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:59:36.800552 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bcce6f-37b5-486b-9011-787391e16adb admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:59:36.802035 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-55bcce6f-37b5-486b-9011-787391e16adb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:59:36.861506 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bcce6f-37b5-486b-9011-787391e16adb admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:59:36.913580 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1c05cf-2721-4b9d-8d83-326ade1ca3f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:59:36.914236 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1c05cf-2721-4b9d-8d83-326ade1ca3f8 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:59:36.914523 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1c05cf-2721-4b9d-8d83-326ade1ca3f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:59:36.914801 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1c05cf-2721-4b9d-8d83-326ade1ca3f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:59:36.916620 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-9b1c05cf-2721-4b9d-8d83-326ade1ca3f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:59:37.003117 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1c05cf-2721-4b9d-8d83-326ade1ca3f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:59:37.004608 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1c05cf-2721-4b9d-8d83-326ade1ca3f8 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:59:37.029313 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1462/2901] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:59:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:59:38.226307 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa330e56-1b48-43b4-b487-11b3f2b19734 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:59:38.227703 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-aa330e56-1b48-43b4-b487-11b3f2b19734 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:59:38.240664 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa330e56-1b48-43b4-b487-11b3f2b19734 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:59:38.244185 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f034ec53-4fbf-4e3e-8d93-0ef9b37f165e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:59:38.244856 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f034ec53-4fbf-4e3e-8d93-0ef9b37f165e service neutron] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:59:38.245141 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f034ec53-4fbf-4e3e-8d93-0ef9b37f165e service neutron] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:59:38.245419 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f034ec53-4fbf-4e3e-8d93-0ef9b37f165e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:59:38.247154 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-f034ec53-4fbf-4e3e-8d93-0ef9b37f165e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:59:38.260495 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f034ec53-4fbf-4e3e-8d93-0ef9b37f165e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:59:38.261961 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f034ec53-4fbf-4e3e-8d93-0ef9b37f165e service neutron] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:59:38.273386 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1440/2902] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:59:38 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:59:47.026246 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=83 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:59:48.268466 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=6 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:59:49.533308 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1c05cf-2721-4b9d-8d83-326ade1ca3f8 service neutron] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:59:49.542169 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1c05cf-2721-4b9d-8d83-326ade1ca3f8 service neutron] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:59:49.546780 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222025df-2e70-45b2-b4b0-f3315fb8813c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94416103f6245888214a997b014f47a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:59:49.547430 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222025df-2e70-45b2-b4b0-f3315fb8813c service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:59:49.547716 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222025df-2e70-45b2-b4b0-f3315fb8813c service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:59:49.547995 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222025df-2e70-45b2-b4b0-f3315fb8813c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:59:49.550052 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-222025df-2e70-45b2-b4b0-f3315fb8813c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:59:49.613647 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222025df-2e70-45b2-b4b0-f3315fb8813c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:59:49.615097 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222025df-2e70-45b2-b4b0-f3315fb8813c service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:59:49.624875 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1463/2903] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 17:59:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:59:56.192846 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f034ec53-4fbf-4e3e-8d93-0ef9b37f165e service neutron] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:59:56.194383 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-f034ec53-4fbf-4e3e-8d93-0ef9b37f165e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:59:56.208503 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f034ec53-4fbf-4e3e-8d93-0ef9b37f165e service neutron] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:59:56.213104 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c279f8-a172-4612-bd99-9bb0a47f0190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:59:56.213974 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c279f8-a172-4612-bd99-9bb0a47f0190 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:59:56.214262 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c279f8-a172-4612-bd99-9bb0a47f0190 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:59:56.214537 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c279f8-a172-4612-bd99-9bb0a47f0190 admin admin] PATH_INFO: `/v3/users/31e04db1b0f6444e9a884a0e58ba38da` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:59:56.215763 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c279f8-a172-4612-bd99-9bb0a47f0190 admin admin] RBAC: Authorizing `identity:delete_user(user_id=31e04db1b0f6444e9a884a0e58ba38da)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:59:56.218581 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c279f8-a172-4612-bd99-9bb0a47f0190 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:59:56.264866 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-e2c279f8-a172-4612-bd99-9bb0a47f0190 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31e04db1b0f6444e9a884a0e58ba38da'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:59:56.265509 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-e2c279f8-a172-4612-bd99-9bb0a47f0190 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31e04db1b0f6444e9a884a0e58ba38da'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:59:56.273263 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-e2c279f8-a172-4612-bd99-9bb0a47f0190 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31e04db1b0f6444e9a884a0e58ba38da'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:59:56.273970 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-e2c279f8-a172-4612-bd99-9bb0a47f0190 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31e04db1b0f6444e9a884a0e58ba38da'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:59:56.286753 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-e2c279f8-a172-4612-bd99-9bb0a47f0190 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31e04db1b0f6444e9a884a0e58ba38da'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:59:56.300197 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1441/2904] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jun 16 17:59:56 2022] DELETE /identity/v3/users/31e04db1b0f6444e9a884a0e58ba38da => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:59:56.473837 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222025df-2e70-45b2-b4b0-f3315fb8813c service neutron] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:59:56.487096 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222025df-2e70-45b2-b4b0-f3315fb8813c service neutron] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:59:56.551411 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06416af-711b-4f07-a9cb-2cfe88fbd0b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:59:56.552315 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06416af-711b-4f07-a9cb-2cfe88fbd0b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:59:56.552625 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06416af-711b-4f07-a9cb-2cfe88fbd0b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:59:56.553020 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06416af-711b-4f07-a9cb-2cfe88fbd0b7 admin admin] PATH_INFO: `/v3/projects/9c249c315494483892639b1213d4bc5d` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:59:56.554229 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06416af-711b-4f07-a9cb-2cfe88fbd0b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c249c315494483892639b1213d4bc5d)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:59:56.556190 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06416af-711b-4f07-a9cb-2cfe88fbd0b7 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:59:56.606407 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-c06416af-711b-4f07-a9cb-2cfe88fbd0b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c249c315494483892639b1213d4bc5d', 'request_id': 'req-c06416af-711b-4f07-a9cb-2cfe88fbd0b7'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:59:56.615089 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-c06416af-711b-4f07-a9cb-2cfe88fbd0b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c249c315494483892639b1213d4bc5d', 'request_id': 'req-c06416af-711b-4f07-a9cb-2cfe88fbd0b7'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:59:56.621664 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-c06416af-711b-4f07-a9cb-2cfe88fbd0b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c249c315494483892639b1213d4bc5d', 'request_id': 'req-c06416af-711b-4f07-a9cb-2cfe88fbd0b7'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:59:56.632722 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-c06416af-711b-4f07-a9cb-2cfe88fbd0b7 admin admin] The token cache is being invalidate because project 9c249c315494483892639b1213d4bc5d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:59:56.633016 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-c06416af-711b-4f07-a9cb-2cfe88fbd0b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:59:56.633743 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-c06416af-711b-4f07-a9cb-2cfe88fbd0b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:59:56.635758 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1464/2905] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jun 16 17:59:56 2022] DELETE /identity/v3/projects/9c249c315494483892639b1213d4bc5d => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:59:56.640589 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c279f8-a172-4612-bd99-9bb0a47f0190 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:59:56.642043 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-e2c279f8-a172-4612-bd99-9bb0a47f0190 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:59:56.655753 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c279f8-a172-4612-bd99-9bb0a47f0190 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:59:56.712454 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9f1831-6620-4738-a578-b958556ea752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:59:56.713345 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9f1831-6620-4738-a578-b958556ea752 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:59:56.713613 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9f1831-6620-4738-a578-b958556ea752 admin admin] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:59:56.713919 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9f1831-6620-4738-a578-b958556ea752 admin admin] PATH_INFO: `/v3/users/a6da6754c62c4da6bb4a264456d2cf01` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:59:56.715093 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9f1831-6620-4738-a578-b958556ea752 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6da6754c62c4da6bb4a264456d2cf01)` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:59:56.718243 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9f1831-6620-4738-a578-b958556ea752 admin admin] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:59:56.765777 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-1b9f1831-6620-4738-a578-b958556ea752 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6da6754c62c4da6bb4a264456d2cf01'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:59:56.766433 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-1b9f1831-6620-4738-a578-b958556ea752 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6da6754c62c4da6bb4a264456d2cf01'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:59:56.773905 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-1b9f1831-6620-4738-a578-b958556ea752 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6da6754c62c4da6bb4a264456d2cf01'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:59:56.774524 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-1b9f1831-6620-4738-a578-b958556ea752 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6da6754c62c4da6bb4a264456d2cf01'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:59:56.787322 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.notifications [None req-1b9f1831-6620-4738-a578-b958556ea752 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6da6754c62c4da6bb4a264456d2cf01'} {{(pid=80357) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:59:56.789307 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:59:56.801428 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1442/2906] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jun 16 17:59:56 2022] DELETE /identity/v3/users/a6da6754c62c4da6bb4a264456d2cf01 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:59:56.973954 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06416af-711b-4f07-a9cb-2cfe88fbd0b7 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:59:56.975439 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-c06416af-711b-4f07-a9cb-2cfe88fbd0b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:59:56.988996 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06416af-711b-4f07-a9cb-2cfe88fbd0b7 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:59:57.010541 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:59:57.050526 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa9365e-3e42-48f2-a6e9-f83181020f6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a220316b144048ad50f0affd87fa43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f64791acc5674278bd1baac8430d9719', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:59:57.051409 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa9365e-3e42-48f2-a6e9-f83181020f6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:59:57.051691 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa9365e-3e42-48f2-a6e9-f83181020f6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:59:57.051966 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa9365e-3e42-48f2-a6e9-f83181020f6b admin admin] PATH_INFO: `/v3/projects/d7254274d5fd4e4c88d98ebc4ad9b47a` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:59:57.053198 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa9365e-3e42-48f2-a6e9-f83181020f6b admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7254274d5fd4e4c88d98ebc4ad9b47a)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:59:57.055100 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa9365e-3e42-48f2-a6e9-f83181020f6b admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:59:57.102637 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-afa9365e-3e42-48f2-a6e9-f83181020f6b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7254274d5fd4e4c88d98ebc4ad9b47a', 'request_id': 'req-afa9365e-3e42-48f2-a6e9-f83181020f6b'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:59:57.110989 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-afa9365e-3e42-48f2-a6e9-f83181020f6b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7254274d5fd4e4c88d98ebc4ad9b47a', 'request_id': 'req-afa9365e-3e42-48f2-a6e9-f83181020f6b'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:59:57.117487 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-afa9365e-3e42-48f2-a6e9-f83181020f6b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7254274d5fd4e4c88d98ebc4ad9b47a', 'request_id': 'req-afa9365e-3e42-48f2-a6e9-f83181020f6b'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:59:57.128339 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-afa9365e-3e42-48f2-a6e9-f83181020f6b admin admin] The token cache is being invalidate because project d7254274d5fd4e4c88d98ebc4ad9b47a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:59:57.128659 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-afa9365e-3e42-48f2-a6e9-f83181020f6b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:59:57.129281 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-afa9365e-3e42-48f2-a6e9-f83181020f6b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:59:57.130824 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1465/2907] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jun 16 17:59:56 2022] DELETE /identity/v3/projects/d7254274d5fd4e4c88d98ebc4ad9b47a => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:00:06.800773 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=4 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:00:07.125428 dv-cinder-837542-4-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=40,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:03:07.337327 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9f1831-6620-4738-a578-b958556ea752 admin admin] Authenticating user token {{(pid=80357) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:03:07.338777 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-1b9f1831-6620-4738-a578-b958556ea752 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:03:07.408907 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9f1831-6620-4738-a578-b958556ea752 admin admin] Validating token access rules against request {{(pid=80357) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:03:07.466713 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e813f3-68f2-4b6b-a380-314cd9150bef service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0570d8d9508d4546ae897276f7d181d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb87f2f141451a8c6ed832db82be18', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80357) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:03:07.467359 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e813f3-68f2-4b6b-a380-314cd9150bef service placement] REQUEST_METHOD: `GET` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:03:07.467645 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e813f3-68f2-4b6b-a380-314cd9150bef service placement] SCRIPT_NAME: `/identity` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:03:07.467918 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e813f3-68f2-4b6b-a380-314cd9150bef service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80357) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:03:07.470333 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.fernet_utils [None req-50e813f3-68f2-4b6b-a380-314cd9150bef service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80357) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:03:07.481495 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e813f3-68f2-4b6b-a380-314cd9150bef service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:03:07.483174 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e813f3-68f2-4b6b-a380-314cd9150bef service placement] RBAC: Authorization granted {{(pid=80357) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:03:07.493585 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: [pid: 80357|app: 0|req: 1443/2908] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jun 16 18:03:07 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:03:17.489075 dv-cinder-837542-4-iscsi devstack@keystone.service[80357]: DEBUG dbcounter [-] [80357] Writing DB stats keystone:SELECT=58 {{(pid=80357) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}